2022-12-01T09:28:30.763 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-12-01T09:28:30.765 DEBUG:teuthology.run:Teuthology command: teuthology --name khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --owner scheduled_khiremat@teuthology --archive /home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090974 --verbose -- /home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090974/orig.config.yaml 2022-12-01T09:28:30.816 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-12-01T09:28:30.893 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090974 branch: wip-khiremat-47893-test-mgr-finisher-thread-5 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7090974' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-khiremat-47893-test-mgr-finisher-thread-5 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-khiremat-47893-test-mgr-finisher-thread-5 sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 owner: scheduled_khiremat@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 sleep_before_teardown: 0 suite: fs:volumes suite_branch: wip-khiremat-47893-test-mgr-finisher-thread-5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f6922ca21f63dad2b198f92826f75d268f12fc97/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCC4yytdRLVQkZ4snbnZWiEij3REBJ1sE4lgmo1QdW3ofIEzQouVkQVZzZ6DL0KEOyr1LRpMldVazXwvPuuSQ1s= smithi085.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ7ZZv9k+3flOBXaHgwJy0MWsDqBrqkE+tGtcRaq7K3AlfFqCqg6/irFVeiXAnHIERXBnwt2HfwDt8tsjgemxGN6pCphEas8ApcMObbcNFIxXKb2VOPfAd7/7K8FrcQD8fDUodOt4YTDpL+3hc2+BR2b08neNjBCKn+gL3/3dYEAt/3IJSVFbwZyU2VKlqlCFwA0cszmgXAxx4r9ZnMEfxtttiiwKOm5jNNJvwQvbJoxt2ecL12DAYtUBaYunggGraZvSNJgiN/vEy6XHNdhLPuBwgUTpw0tL5LZCHilphTWSDqYXJmOgBohAIJitB1poDkSRQF+ORVlpFy/v08mbScYHcGyyigMkERheSfpAs7AmFR8dQpZ2RP820OlRw78Zy0RaX7Dz0VzFiF/WuE690NtN/SSJz+srXhJSCO9vTNLM4RQ12M0/HcNha/L2F7nJyie9om70QDajcwtoLBziHOc3gWm71ge901Zud7RuyOOH3Jd56m3oD+d7OgviOYEM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-24_05:02:42 tube: smithi user: khiremat verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23126 2022-12-01T09:28:30.894 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f6922ca21f63dad2b198f92826f75d268f12fc97/qa; will attempt to use it 2022-12-01T09:28:30.895 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f6922ca21f63dad2b198f92826f75d268f12fc97/qa/tasks 2022-12-01T09:28:30.895 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-12-01T09:28:30.897 INFO:teuthology.task.internal:Checking packages... 2022-12-01T09:28:30.922 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'f6922ca21f63dad2b198f92826f75d268f12fc97' 2022-12-01T09:28:30.922 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-12-01T09:28:30.923 INFO:teuthology.packaging:ref: None 2022-12-01T09:28:30.923 INFO:teuthology.packaging:tag: None 2022-12-01T09:28:30.924 INFO:teuthology.packaging:branch: wip-khiremat-47893-test-mgr-finisher-thread-5 2022-12-01T09:28:30.924 INFO:teuthology.packaging:sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:28:30.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-khiremat-47893-test-mgr-finisher-thread-5 2022-12-01T09:28:31.180 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-676.gf6922ca2 2022-12-01T09:28:31.180 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-12-01T09:28:31.187 INFO:teuthology.task.internal:no buildpackages task found 2022-12-01T09:28:31.187 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-12-01T09:28:31.193 INFO:teuthology.task.internal:Saving configuration 2022-12-01T09:28:31.212 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-12-01T09:28:31.218 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-12-01T09:28:31.245 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090974', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-12-01 09:23:24.752223', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCC4yytdRLVQkZ4snbnZWiEij3REBJ1sE4lgmo1QdW3ofIEzQouVkQVZzZ6DL0KEOyr1LRpMldVazXwvPuuSQ1s='} 2022-12-01T09:28:31.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi085.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090974', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-12-01 09:23:24.750667', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVgkUiaZY81dtpCjYeLQRV14x2c68fxRAWlN0uG6r1zF+YE8RReR3jhrV2ffCCIa8uX+JkJx2yopjYjuuoPXb4='} 2022-12-01T09:28:31.271 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-12-01T09:28:31.277 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-12-01T09:28:31.278 INFO:teuthology.task.internal:roles: ubuntu@smithi085.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-12-01T09:28:31.278 INFO:teuthology.run_tasks:Running task console_log... 2022-12-01T09:28:31.369 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7a6a62e8c8>, signals=[15]) 2022-12-01T09:28:31.370 INFO:teuthology.run_tasks:Running task internal.connect... 2022-12-01T09:28:31.376 INFO:teuthology.task.internal:Opening connections... 2022-12-01T09:28:31.377 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2022-12-01T09:28:31.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-12-01T09:28:31.458 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi085.front.sepia.ceph.com 2022-12-01T09:28:31.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-12-01T09:28:31.539 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-12-01T09:28:31.546 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2022-12-01T09:28:31.568 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2022-12-01T09:28:31.568 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2022-12-01T09:28:31.629 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Red Hat Enterprise Linux" 2022-12-01T09:28:31.629 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8.6 (Ootpa)" 2022-12-01T09:28:31.629 INFO:teuthology.orchestra.run.smithi002.stdout:ID="rhel" 2022-12-01T09:28:31.630 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="fedora" 2022-12-01T09:28:31.630 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8.6" 2022-12-01T09:28:31.630 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2022-12-01T09:28:31.630 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-12-01T09:28:31.631 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2022-12-01T09:28:31.631 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-12-01T09:28:31.631 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.redhat.com/" 2022-12-01T09:28:31.631 INFO:teuthology.orchestra.run.smithi002.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-12-01T09:28:31.632 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-12-01T09:28:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:28:31.632 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-12-01T09:28:31.632 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-12-01T09:28:31.633 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-12-01T09:28:31.633 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-12-01T09:28:31.634 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2022-12-01T09:28:31.656 DEBUG:teuthology.orchestra.run.smithi085:> uname -m 2022-12-01T09:28:31.676 INFO:teuthology.orchestra.run.smithi085.stdout:x86_64 2022-12-01T09:28:31.677 DEBUG:teuthology.orchestra.run.smithi085:> cat /etc/os-release 2022-12-01T09:28:31.736 INFO:teuthology.orchestra.run.smithi085.stdout:NAME="Red Hat Enterprise Linux" 2022-12-01T09:28:31.736 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION="8.6 (Ootpa)" 2022-12-01T09:28:31.736 INFO:teuthology.orchestra.run.smithi085.stdout:ID="rhel" 2022-12-01T09:28:31.737 INFO:teuthology.orchestra.run.smithi085.stdout:ID_LIKE="fedora" 2022-12-01T09:28:31.737 INFO:teuthology.orchestra.run.smithi085.stdout:VERSION_ID="8.6" 2022-12-01T09:28:31.737 INFO:teuthology.orchestra.run.smithi085.stdout:PLATFORM_ID="platform:el8" 2022-12-01T09:28:31.737 INFO:teuthology.orchestra.run.smithi085.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-12-01T09:28:31.738 INFO:teuthology.orchestra.run.smithi085.stdout:ANSI_COLOR="0;31" 2022-12-01T09:28:31.738 INFO:teuthology.orchestra.run.smithi085.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-12-01T09:28:31.738 INFO:teuthology.orchestra.run.smithi085.stdout:HOME_URL="https://www.redhat.com/" 2022-12-01T09:28:31.738 INFO:teuthology.orchestra.run.smithi085.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-12-01T09:28:31.739 INFO:teuthology.orchestra.run.smithi085.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-12-01T09:28:31.739 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:28:31.739 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-12-01T09:28:31.739 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-12-01T09:28:31.739 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-12-01T09:28:31.740 INFO:teuthology.orchestra.run.smithi085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-12-01T09:28:31.740 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2022-12-01T09:28:31.759 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-12-01T09:28:31.770 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-12-01T09:28:31.775 INFO:teuthology.task.internal:Checking for old test directory... 2022-12-01T09:28:31.776 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2022-12-01T09:28:31.779 DEBUG:teuthology.orchestra.run.smithi085:> test '!' -e /home/ubuntu/cephtest 2022-12-01T09:28:31.797 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-12-01T09:28:31.805 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-12-01T09:28:31.805 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2022-12-01T09:28:31.839 DEBUG:teuthology.orchestra.run.smithi085:> test -z $(ls -A /var/lib/ceph) 2022-12-01T09:28:31.867 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-12-01T09:28:31.943 INFO:teuthology.run_tasks:Running task kernel... 2022-12-01T09:28:31.957 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2022-12-01T09:28:31.958 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2022-12-01T09:28:31.958 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2022-12-01T09:28:31.959 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2022-12-01T09:28:31.960 DEBUG:teuthology.orchestra.run.smithi085:> test -f /run/.containerenv -o -f /.dockerenv 2022-12-01T09:28:31.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-12-01T09:28:31.978 DEBUG:teuthology.orchestra.run.smithi085:> uname -r 2022-12-01T09:28:31.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-12-01T09:28:31.980 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2022-12-01T09:28:32.038 INFO:teuthology.orchestra.run.smithi085.stdout:4.18.0-372.9.1.el8.x86_64 2022-12-01T09:28:32.038 INFO:teuthology.task.kernel:Running kernel on smithi085: 4.18.0-372.9.1.el8.x86_64 2022-12-01T09:28:32.039 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum install -y kernel 2022-12-01T09:28:32.041 INFO:teuthology.orchestra.run.smithi002.stdout:4.18.0-372.9.1.el8.x86_64 2022-12-01T09:28:32.041 INFO:teuthology.task.kernel:Running kernel on smithi002: 4.18.0-372.9.1.el8.x86_64 2022-12-01T09:28:32.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2022-12-01T09:28:32.968 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-12-01T09:28:32.968 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2022-12-01T09:28:33.060 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2022-12-01T09:28:33.061 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2022-12-01T09:28:33.072 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:28:33.072 INFO:teuthology.orchestra.run.smithi085.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:28:33.073 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:28:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:28:33.148 INFO:teuthology.orchestra.run.smithi002.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:28:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:28:35.956 INFO:teuthology.orchestra.run.smithi085.stdout:Copr repo for python3-asyncssh owned by ceph 2.0 kB/s | 3.5 kB 00:01 2022-12-01T09:28:35.958 INFO:teuthology.orchestra.run.smithi002.stdout:Copr repo for python3-asyncssh owned by ceph 2.1 kB/s | 3.5 kB 00:01 2022-12-01T09:28:38.383 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 13 MB 00:02 2022-12-01T09:28:40.239 INFO:teuthology.orchestra.run.smithi085.stdout:Extra Packages for Enterprise Linux 3.1 MB/s | 13 MB 00:04 2022-12-01T09:28:41.525 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 365 kB/s | 24 kB 00:00 2022-12-01T09:28:42.245 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-12-01T09:28:42.302 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2022-12-01T09:28:42.303 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2022-12-01T09:28:42.304 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2022-12-01T09:28:42.423 DEBUG:teuthology.orchestra.run.smithi002:> echo no | sudo yum reinstall kernel || true 2022-12-01T09:28:42.814 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2022-12-01T09:28:42.814 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2022-12-01T09:28:42.860 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:28:42.860 INFO:teuthology.orchestra.run.smithi002.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:28:42.861 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:28:43.117 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:02 ago on Thu 01 Dec 2022 09:28:41 AM UTC. 2022-12-01T09:28:43.161 INFO:teuthology.orchestra.run.smithi002.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-12-01T09:28:43.209 INFO:teuthology.orchestra.run.smithi002.stderr:Error: No packages marked for reinstall. 2022-12-01T09:28:43.214 INFO:teuthology.orchestra.run.smithi085.stdout:lab-extras 491 kB/s | 24 kB 00:00 2022-12-01T09:28:43.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum reinstall -y kernel || true 2022-12-01T09:28:43.607 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2022-12-01T09:28:43.608 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2022-12-01T09:28:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:28:43.651 INFO:teuthology.orchestra.run.smithi002.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:28:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:28:43.877 INFO:teuthology.orchestra.run.smithi085.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-12-01T09:28:43.903 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:02 ago on Thu 01 Dec 2022 09:28:41 AM UTC. 2022-12-01T09:28:43.930 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2022-12-01T09:28:43.931 INFO:teuthology.orchestra.run.smithi085.stdout:Nothing to do. 2022-12-01T09:28:43.932 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2022-12-01T09:28:43.945 INFO:teuthology.orchestra.run.smithi002.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-12-01T09:28:43.978 INFO:teuthology.orchestra.run.smithi002.stderr:Error: No packages marked for reinstall. 2022-12-01T09:28:44.026 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel --last | head -n 1 2022-12-01T09:28:44.061 DEBUG:teuthology.orchestra.run.smithi085:> echo no | sudo yum reinstall kernel || true 2022-12-01T09:28:44.108 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-12-01T09:28:44.110 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-12-01T09:28:44.110 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-12-01T09:28:44.111 DEBUG:teuthology.parallel:result is None 2022-12-01T09:28:44.392 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-12-01T09:28:44.393 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2022-12-01T09:28:44.436 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:28:44.436 INFO:teuthology.orchestra.run.smithi085.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:28:44.436 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:28:44.676 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:01 ago on Thu 01 Dec 2022 09:28:43 AM UTC. 2022-12-01T09:28:44.728 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-12-01T09:28:44.775 INFO:teuthology.orchestra.run.smithi085.stderr:Error: No packages marked for reinstall. 2022-12-01T09:28:44.820 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum reinstall -y kernel || true 2022-12-01T09:28:45.150 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-12-01T09:28:45.150 INFO:teuthology.orchestra.run.smithi085.stdout:Unable to read consumer identity 2022-12-01T09:28:45.192 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:28:45.192 INFO:teuthology.orchestra.run.smithi085.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:28:45.192 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:28:45.427 INFO:teuthology.orchestra.run.smithi085.stdout:Last metadata expiration check: 0:00:02 ago on Thu 01 Dec 2022 09:28:43 AM UTC. 2022-12-01T09:28:45.465 INFO:teuthology.orchestra.run.smithi085.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-12-01T09:28:45.624 INFO:teuthology.orchestra.run.smithi085.stderr:Error: No packages marked for reinstall. 2022-12-01T09:28:45.672 DEBUG:teuthology.orchestra.run.smithi085:> rpm -q kernel --last | head -n 1 2022-12-01T09:28:45.751 INFO:teuthology.orchestra.run.smithi085.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-12-01T09:28:45.753 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-12-01T09:28:45.753 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-12-01T09:28:45.754 DEBUG:teuthology.parallel:result is None 2022-12-01T09:28:45.754 INFO:teuthology.run_tasks:Running task internal.base... 2022-12-01T09:28:45.761 INFO:teuthology.task.internal:Creating test directory... 2022-12-01T09:28:45.762 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-12-01T09:28:45.765 DEBUG:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-12-01T09:28:45.784 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-12-01T09:28:45.791 INFO:teuthology.run_tasks:Running task internal.archive... 2022-12-01T09:28:45.797 INFO:teuthology.task.internal:Creating archive directory... 2022-12-01T09:28:45.798 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-12-01T09:28:45.826 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-12-01T09:28:45.863 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-12-01T09:28:45.869 INFO:teuthology.task.internal:Enabling coredump saving... 2022-12-01T09:28:45.870 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-12-01T09:28:45.892 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-12-01T09:28:45.931 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-12-01T09:28:45.932 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-12-01T09:28:45.946 INFO:teuthology.orchestra.run.smithi085.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-12-01T09:28:45.947 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-12-01T09:28:45.948 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-12-01T09:28:45.956 INFO:teuthology.task.internal:Configuring sudo... 2022-12-01T09:28:45.957 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-12-01T09:28:45.991 DEBUG:teuthology.orchestra.run.smithi085:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-12-01T09:28:46.020 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-12-01T09:28:46.029 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-12-01T09:28:46.029 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-12-01T09:28:46.062 DEBUG:teuthology.orchestra.run.smithi085:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-12-01T09:28:46.079 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-12-01T09:28:46.163 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-12-01T09:28:46.216 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-12-01T09:28:46.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-12-01T09:28:46.344 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:28:46.344 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-12-01T09:28:46.411 DEBUG:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-12-01T09:28:46.456 DEBUG:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-12-01T09:28:46.507 DEBUG:teuthology.orchestra.run.smithi085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-12-01T09:28:46.591 DEBUG:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-12-01T09:28:46.633 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:28:46.634 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-12-01T09:28:46.700 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2022-12-01T09:28:46.703 DEBUG:teuthology.orchestra.run.smithi085:> sudo service rsyslog restart 2022-12-01T09:28:46.750 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-12-01T09:28:46.787 INFO:teuthology.orchestra.run.smithi085.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-12-01T09:28:47.157 INFO:teuthology.run_tasks:Running task internal.timer... 2022-12-01T09:28:47.166 INFO:teuthology.task.internal:Starting timer... 2022-12-01T09:28:47.166 INFO:teuthology.run_tasks:Running task pcp... 2022-12-01T09:28:47.178 INFO:teuthology.run_tasks:Running task selinux... 2022-12-01T09:28:47.186 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2022-12-01T09:28:47.187 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2022-12-01T09:28:47.237 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2022-12-01T09:28:47.239 DEBUG:teuthology.orchestra.run.smithi085:> sudo service auditd rotate 2022-12-01T09:28:47.300 INFO:teuthology.orchestra.run.smithi085.stdout:Rotating logs: 2022-12-01T09:28:47.302 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-12-01T09:28:47.302 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2022-12-01T09:28:47.321 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2022-12-01T09:28:47.322 DEBUG:teuthology.orchestra.run.smithi085:> /usr/sbin/getenforce 2022-12-01T09:28:47.357 INFO:teuthology.orchestra.run.smithi085.stdout:Permissive 2022-12-01T09:28:47.358 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi085.front.sepia.ceph.com': 'permissive'} 2022-12-01T09:28:47.358 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-12-01T09:28:47.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-12-01T09:28:47.390 DEBUG:teuthology.orchestra.run.smithi085:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-12-01T09:28:47.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-12-01T09:28:47.426 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-12-01T09:28:47.427 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2022-12-01T09:28:47.457 DEBUG:teuthology.orchestra.run.smithi085:> sudo /usr/sbin/setenforce permissive 2022-12-01T09:28:47.494 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-12-01T09:28:47.504 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2022-12-01T09:28:47.505 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-12-01T09:28:47.532 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-12-01T09:28:47.534 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi085.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-12-01T09:36:47.439 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi085.front.sepia.ceph.com')] 2022-12-01T09:36:47.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2022-12-01T09:36:47.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-12-01T09:36:47.528 DEBUG:teuthology.orchestra.run.smithi002:> true 2022-12-01T09:36:47.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi085.front.sepia.ceph.com' 2022-12-01T09:36:47.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-12-01T09:36:47.676 DEBUG:teuthology.orchestra.run.smithi085:> true 2022-12-01T09:36:47.749 INFO:teuthology.run_tasks:Running task clock... 2022-12-01T09:36:47.761 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-12-01T09:36:47.762 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-12-01T09:36:47.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-12-01T09:36:47.767 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-12-01T09:36:47.768 DEBUG:teuthology.orchestra.run.smithi085:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-12-01T09:36:47.799 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-12-01T09:36:47.814 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-12-01T09:36:47.821 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-12-01T09:36:47.835 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-12-01T09:36:47.888 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2022-12-01T09:36:47.899 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2022-12-01T09:36:47.904 INFO:teuthology.orchestra.run.smithi085.stderr:sudo: ntpd: command not found 2022-12-01T09:36:47.914 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-12-01T09:36:47.916 INFO:teuthology.orchestra.run.smithi085.stdout:506 Cannot talk to daemon 2022-12-01T09:36:47.929 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-12-01T09:36:47.930 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-12-01T09:36:47.944 INFO:teuthology.orchestra.run.smithi085.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-12-01T09:36:48.001 INFO:teuthology.orchestra.run.smithi002.stderr:bash: ntpq: command not found 2022-12-01T09:36:48.005 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-12-01T09:36:48.005 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2022-12-01T09:36:48.005 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:36:48.006 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:36:48.006 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:36:48.006 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:36:48.017 INFO:teuthology.orchestra.run.smithi085.stderr:bash: ntpq: command not found 2022-12-01T09:36:48.021 INFO:teuthology.orchestra.run.smithi085.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-12-01T09:36:48.021 INFO:teuthology.orchestra.run.smithi085.stdout:=============================================================================== 2022-12-01T09:36:48.021 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:36:48.021 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:36:48.021 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:36:48.022 INFO:teuthology.orchestra.run.smithi085.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:36:48.023 INFO:teuthology.run_tasks:Running task install... 2022-12-01T09:36:48.032 DEBUG:teuthology.task.install:project ceph 2022-12-01T09:36:48.032 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}} 2022-12-01T09:36:48.033 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'} 2022-12-01T09:36:48.033 INFO:teuthology.task.install:Using flavor: default 2022-12-01T09:36:48.046 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-12-01T09:36:48.047 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-12-01T09:36:48.048 DEBUG:teuthology.orchestra.run.smithi002:> sudo lsb_release -is 2022-12-01T09:36:48.091 INFO:teuthology.orchestra.run.smithi002.stdout:RedHatEnterprise 2022-12-01T09:36:48.092 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:36:48.093 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-12-01T09:36:48.094 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97', 'tag': None, 'wait_for_package': False} 2022-12-01T09:36:48.094 DEBUG:teuthology.orchestra.run.smithi002:> sudo lsb_release -is 2022-12-01T09:36:48.135 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2022-12-01T09:36:48.137 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:36:48.137 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97', 'tag': None, 'wait_for_package': False} 2022-12-01T09:36:48.138 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-12-01T09:36:48.174 INFO:teuthology.orchestra.run.smithi002.stdout:RedHatEnterprise 2022-12-01T09:36:48.176 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:36:48.177 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:36:48.221 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2022-12-01T09:36:48.222 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:36:48.222 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:36:48.488 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/ 2022-12-01T09:36:48.489 INFO:teuthology.task.install.rpm:Package version is 18.0.0-676.gf6922ca2 2022-12-01T09:36:48.539 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/ 2022-12-01T09:36:48.540 INFO:teuthology.task.install.rpm:Package version is 18.0.0-676.gf6922ca2 2022-12-01T09:36:48.735 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-12-01T09:36:48.735 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:36:48.736 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-12-01T09:36:48.766 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-12-01T09:36:48.766 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f6922ca21f63dad2b198f92826f75d268f12fc97/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-12-01T09:36:48.789 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-12-01T09:36:48.789 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:36:48.789 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-12-01T09:36:48.817 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-12-01T09:36:48.818 DEBUG:teuthology.orchestra.run.smithi085:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f6922ca21f63dad2b198f92826f75d268f12fc97/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-12-01T09:36:48.836 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-12-01T09:36:48.888 DEBUG:teuthology.orchestra.run.smithi085:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-12-01T09:36:48.916 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-12-01T09:36:48.966 DEBUG:teuthology.orchestra.run.smithi085:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-12-01T09:36:48.983 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2022-12-01T09:36:48.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2022-12-01T09:36:49.033 INFO:teuthology.orchestra.run.smithi085.stdout:check_obsoletes = 1 2022-12-01T09:36:49.034 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum clean all 2022-12-01T09:36:49.376 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:49.407 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:49.617 INFO:teuthology.orchestra.run.smithi085.stdout:54 files removed 2022-12-01T09:36:49.659 INFO:teuthology.orchestra.run.smithi002.stdout:54 files removed 2022-12-01T09:36:49.664 DEBUG:teuthology.orchestra.run.smithi085:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-12-01T09:36:49.706 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-12-01T09:36:49.995 INFO:teuthology.orchestra.run.smithi085.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:50.056 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:50.909 INFO:teuthology.orchestra.run.smithi085.stdout:ceph packages for x86_64 150 kB/s | 81 kB 00:00 2022-12-01T09:36:50.919 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 155 kB/s | 81 kB 00:00 2022-12-01T09:36:51.443 INFO:teuthology.orchestra.run.smithi085.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2022-12-01T09:36:51.470 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2022-12-01T09:36:51.969 INFO:teuthology.orchestra.run.smithi085.stdout:ceph source packages 3.0 kB/s | 1.5 kB 00:00 2022-12-01T09:36:51.996 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 3.0 kB/s | 1.5 kB 00:00 2022-12-01T09:36:53.594 INFO:teuthology.orchestra.run.smithi085.stdout:Copr repo for python3-asyncssh owned by ceph 2.2 kB/s | 3.5 kB 00:01 2022-12-01T09:36:53.621 INFO:teuthology.orchestra.run.smithi002.stdout:Copr repo for python3-asyncssh owned by ceph 2.2 kB/s | 3.5 kB 00:01 2022-12-01T09:36:59.902 INFO:teuthology.orchestra.run.smithi085.stdout:Extra Packages for Enterprise Linux 2.1 MB/s | 13 MB 00:06 2022-12-01T09:37:00.646 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 13 MB 00:07 2022-12-01T09:37:02.919 INFO:teuthology.orchestra.run.smithi085.stdout:lab-extras 283 kB/s | 24 kB 00:00 2022-12-01T09:37:03.779 INFO:teuthology.orchestra.run.smithi085.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 67 MB/s | 53 MB 00:00 2022-12-01T09:37:03.904 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 159 kB/s | 24 kB 00:00 2022-12-01T09:37:04.731 INFO:teuthology.orchestra.run.smithi002.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 70 MB/s | 53 MB 00:00 2022-12-01T09:37:11.813 INFO:teuthology.orchestra.run.smithi085.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 66 MB/s | 48 MB 00:00 2022-12-01T09:37:13.415 INFO:teuthology.orchestra.run.smithi002.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 64 MB/s | 48 MB 00:00 2022-12-01T09:37:22.954 INFO:teuthology.orchestra.run.smithi085.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-12-01T09:37:22.955 INFO:teuthology.orchestra.run.smithi085.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-12-01T09:37:22.970 INFO:teuthology.orchestra.run.smithi085.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-12-01T09:37:22.971 INFO:teuthology.orchestra.run.smithi085.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-12-01T09:37:22.972 INFO:teuthology.orchestra.run.smithi085.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-12-01T09:37:22.974 INFO:teuthology.orchestra.run.smithi085.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-12-01T09:37:22.976 INFO:teuthology.orchestra.run.smithi085.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-12-01T09:37:22.979 INFO:teuthology.orchestra.run.smithi085.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-12-01T09:37:22.980 INFO:teuthology.orchestra.run.smithi085.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-12-01T09:37:23.110 INFO:teuthology.orchestra.run.smithi085.stdout:Dependencies resolved. 2022-12-01T09:37:23.122 INFO:teuthology.orchestra.run.smithi085.stdout:=========================================================================================================================== 2022-12-01T09:37:23.122 INFO:teuthology.orchestra.run.smithi085.stdout: Package Arch Version Repository Size 2022-12-01T09:37:23.123 INFO:teuthology.orchestra.run.smithi085.stdout:=========================================================================================================================== 2022-12-01T09:37:23.123 INFO:teuthology.orchestra.run.smithi085.stdout:Installing: 2022-12-01T09:37:23.123 INFO:teuthology.orchestra.run.smithi085.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-12-01T09:37:23.124 INFO:teuthology.orchestra.run.smithi085.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-12-01T09:37:23.124 INFO:teuthology.orchestra.run.smithi085.stdout: ceph x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 6.4 k 2022-12-01T09:37:23.124 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 5.2 M 2022-12-01T09:37:23.124 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-fuse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 845 k 2022-12-01T09:37:23.125 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 165 k 2022-12-01T09:37:23.125 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 1.5 M 2022-12-01T09:37:23.125 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-cephadm noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 118 k 2022-12-01T09:37:23.125 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 1.7 M 2022-12-01T09:37:23.126 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 7.4 M 2022-12-01T09:37:23.126 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-rook noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 47 k 2022-12-01T09:37:23.126 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-radosgw x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 18 M 2022-12-01T09:37:23.126 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-test x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 55 M 2022-12-01T09:37:23.127 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-volume noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 263 k 2022-12-01T09:37:23.127 INFO:teuthology.orchestra.run.smithi085.stdout: cephadm noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 81 k 2022-12-01T09:37:23.127 INFO:teuthology.orchestra.run.smithi085.stdout: cephfs-mirror x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 257 k 2022-12-01T09:37:23.127 INFO:teuthology.orchestra.run.smithi085.stdout: cephfs-top noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 15 k 2022-12-01T09:37:23.128 INFO:teuthology.orchestra.run.smithi085.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-12-01T09:37:23.128 INFO:teuthology.orchestra.run.smithi085.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-12-01T09:37:23.128 INFO:teuthology.orchestra.run.smithi085.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-12-01T09:37:23.128 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs-devel x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 30 k 2022-12-01T09:37:23.129 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 768 k 2022-12-01T09:37:23.129 INFO:teuthology.orchestra.run.smithi085.stdout: librados-devel x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 139 k 2022-12-01T09:37:23.129 INFO:teuthology.orchestra.run.smithi085.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-12-01T09:37:23.129 INFO:teuthology.orchestra.run.smithi085.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-12-01T09:37:23.130 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-12-01T09:37:23.130 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-12-01T09:37:23.130 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-12-01T09:37:23.130 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-12-01T09:37:23.131 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cephfs x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 190 k 2022-12-01T09:37:23.131 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 370 k 2022-12-01T09:37:23.131 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rbd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 364 k 2022-12-01T09:37:23.131 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 107 k 2022-12-01T09:37:23.132 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-fuse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 94 k 2022-12-01T09:37:23.132 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-mirror x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.5 M 2022-12-01T09:37:23.132 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-nbd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 177 k 2022-12-01T09:37:23.132 INFO:teuthology.orchestra.run.smithi085.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-12-01T09:37:23.132 INFO:teuthology.orchestra.run.smithi085.stdout:Upgrading: 2022-12-01T09:37:23.133 INFO:teuthology.orchestra.run.smithi085.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-12-01T09:37:23.133 INFO:teuthology.orchestra.run.smithi085.stdout: librados2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.6 M 2022-12-01T09:37:23.133 INFO:teuthology.orchestra.run.smithi085.stdout: librbd1 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.6 M 2022-12-01T09:37:23.133 INFO:teuthology.orchestra.run.smithi085.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-12-01T09:37:23.134 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-12-01T09:37:23.134 INFO:teuthology.orchestra.run.smithi085.stdout:Installing dependencies: 2022-12-01T09:37:23.134 INFO:teuthology.orchestra.run.smithi085.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-12-01T09:37:23.134 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-common x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 24 M 2022-12-01T09:37:23.134 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-grafana-dashboards noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 23 k 2022-12-01T09:37:23.135 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mds x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 2.3 M 2022-12-01T09:37:23.135 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-modules-core noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 234 k 2022-12-01T09:37:23.135 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.9 M 2022-12-01T09:37:23.135 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-osd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 17 M 2022-12-01T09:37:23.136 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-prometheus-alerts noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 14 k 2022-12-01T09:37:23.136 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-selinux x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 24 k 2022-12-01T09:37:23.136 INFO:teuthology.orchestra.run.smithi085.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-12-01T09:37:23.136 INFO:teuthology.orchestra.run.smithi085.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-12-01T09:37:23.136 INFO:teuthology.orchestra.run.smithi085.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-12-01T09:37:23.137 INFO:teuthology.orchestra.run.smithi085.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-12-01T09:37:23.137 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-12-01T09:37:23.137 INFO:teuthology.orchestra.run.smithi085.stdout: libcephsqlite x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 137 k 2022-12-01T09:37:23.137 INFO:teuthology.orchestra.run.smithi085.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-12-01T09:37:23.138 INFO:teuthology.orchestra.run.smithi085.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-12-01T09:37:23.138 INFO:teuthology.orchestra.run.smithi085.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-12-01T09:37:23.138 INFO:teuthology.orchestra.run.smithi085.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-12-01T09:37:23.138 INFO:teuthology.orchestra.run.smithi085.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-12-01T09:37:23.138 INFO:teuthology.orchestra.run.smithi085.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-12-01T09:37:23.139 INFO:teuthology.orchestra.run.smithi085.stdout: libradosstriper1 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 474 k 2022-12-01T09:37:23.139 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-12-01T09:37:23.139 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 6.5 M 2022-12-01T09:37:23.139 INFO:teuthology.orchestra.run.smithi085.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-12-01T09:37:23.140 INFO:teuthology.orchestra.run.smithi085.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-12-01T09:37:23.140 INFO:teuthology.orchestra.run.smithi085.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-12-01T09:37:23.140 INFO:teuthology.orchestra.run.smithi085.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-12-01T09:37:23.140 INFO:teuthology.orchestra.run.smithi085.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-12-01T09:37:23.140 INFO:teuthology.orchestra.run.smithi085.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-12-01T09:37:23.141 INFO:teuthology.orchestra.run.smithi085.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-12-01T09:37:23.141 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-12-01T09:37:23.141 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-12-01T09:37:23.141 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-12-01T09:37:23.141 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-12-01T09:37:23.142 INFO:teuthology.orchestra.run.smithi085.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-12-01T09:37:23.142 INFO:teuthology.orchestra.run.smithi085.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-12-01T09:37:23.142 INFO:teuthology.orchestra.run.smithi085.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-12-01T09:37:23.142 INFO:teuthology.orchestra.run.smithi085.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-12-01T09:37:23.143 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-12-01T09:37:23.143 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 45 k 2022-12-01T09:37:23.143 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-common x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 109 k 2022-12-01T09:37:23.143 INFO:teuthology.orchestra.run.smithi085.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-12-01T09:37:23.143 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-12-01T09:37:23.144 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-12-01T09:37:23.144 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-12-01T09:37:23.144 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-12-01T09:37:23.144 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-12-01T09:37:23.145 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-12-01T09:37:23.145 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-12-01T09:37:23.145 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-12-01T09:37:23.145 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-12-01T09:37:23.146 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-12-01T09:37:23.146 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-12-01T09:37:23.146 INFO:teuthology.orchestra.run.smithi085.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-12-01T09:37:23.146 INFO:teuthology.orchestra.run.smithi085.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-12-01T09:37:23.147 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-12-01T09:37:23.147 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-12-01T09:37:23.147 INFO:teuthology.orchestra.run.smithi085.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-12-01T09:37:23.147 INFO:teuthology.orchestra.run.smithi085.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-12-01T09:37:23.148 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-12-01T09:37:23.148 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-12-01T09:37:23.148 INFO:teuthology.orchestra.run.smithi085.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-12-01T09:37:23.148 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-12-01T09:37:23.148 INFO:teuthology.orchestra.run.smithi085.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-12-01T09:37:23.149 INFO:teuthology.orchestra.run.smithi085.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-12-01T09:37:23.149 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-12-01T09:37:23.149 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-12-01T09:37:23.149 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-12-01T09:37:23.150 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-12-01T09:37:23.150 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-12-01T09:37:23.150 INFO:teuthology.orchestra.run.smithi085.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-12-01T09:37:23.150 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-12-01T09:37:23.150 INFO:teuthology.orchestra.run.smithi085.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-12-01T09:37:23.151 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-12-01T09:37:23.151 INFO:teuthology.orchestra.run.smithi085.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-12-01T09:37:23.151 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-12-01T09:37:23.151 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-12-01T09:37:23.152 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-12-01T09:37:23.152 INFO:teuthology.orchestra.run.smithi085.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-12-01T09:37:23.152 INFO:teuthology.orchestra.run.smithi085.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-12-01T09:37:23.152 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-12-01T09:37:23.153 INFO:teuthology.orchestra.run.smithi085.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-12-01T09:37:23.153 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-12-01T09:37:23.153 INFO:teuthology.orchestra.run.smithi085.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-12-01T09:37:23.153 INFO:teuthology.orchestra.run.smithi085.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-12-01T09:37:23.154 INFO:teuthology.orchestra.run.smithi085.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-12-01T09:37:23.154 INFO:teuthology.orchestra.run.smithi085.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-12-01T09:37:23.154 INFO:teuthology.orchestra.run.smithi085.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-12-01T09:37:23.154 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-12-01T09:37:23.155 INFO:teuthology.orchestra.run.smithi085.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-12-01T09:37:23.155 INFO:teuthology.orchestra.run.smithi085.stdout:Enabling module streams: 2022-12-01T09:37:23.155 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql 10 2022-12-01T09:37:23.155 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:37:23.156 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction Summary 2022-12-01T09:37:23.156 INFO:teuthology.orchestra.run.smithi085.stdout:=========================================================================================================================== 2022-12-01T09:37:23.156 INFO:teuthology.orchestra.run.smithi085.stdout:Install 128 Packages 2022-12-01T09:37:23.156 INFO:teuthology.orchestra.run.smithi085.stdout:Upgrade 5 Packages 2022-12-01T09:37:23.156 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:37:23.157 INFO:teuthology.orchestra.run.smithi085.stdout:Total download size: 219 M 2022-12-01T09:37:23.158 INFO:teuthology.orchestra.run.smithi085.stdout:Downloading Packages: 2022-12-01T09:37:23.438 INFO:teuthology.orchestra.run.smithi085.stdout:(1/133): ceph-18.0.0-676.gf6922ca2.el8.x86_64.r 21 kB/s | 6.4 kB 00:00 2022-12-01T09:37:24.289 INFO:teuthology.orchestra.run.smithi085.stdout:(2/133): ceph-fuse-18.0.0-676.gf6922ca2.el8.x86 993 kB/s | 845 kB 00:00 2022-12-01T09:37:24.455 INFO:teuthology.orchestra.run.smithi085.stdout:(3/133): ceph-immutable-object-cache-18.0.0-676 998 kB/s | 165 kB 00:00 2022-12-01T09:37:25.204 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-12-01T09:37:25.205 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-12-01T09:37:25.220 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-12-01T09:37:25.221 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-12-01T09:37:25.222 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-12-01T09:37:25.223 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-12-01T09:37:25.225 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-12-01T09:37:25.229 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-12-01T09:37:25.230 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-12-01T09:37:25.378 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2022-12-01T09:37:25.392 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2022-12-01T09:37:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2022-12-01T09:37:25.392 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2022-12-01T09:37:25.393 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2022-12-01T09:37:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-12-01T09:37:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-12-01T09:37:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 6.4 k 2022-12-01T09:37:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 5.2 M 2022-12-01T09:37:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 845 k 2022-12-01T09:37:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 165 k 2022-12-01T09:37:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 1.5 M 2022-12-01T09:37:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 118 k 2022-12-01T09:37:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 1.7 M 2022-12-01T09:37:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 7.4 M 2022-12-01T09:37:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 47 k 2022-12-01T09:37:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 18 M 2022-12-01T09:37:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 55 M 2022-12-01T09:37:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 263 k 2022-12-01T09:37:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 81 k 2022-12-01T09:37:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 257 k 2022-12-01T09:37:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 15 k 2022-12-01T09:37:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-12-01T09:37:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-12-01T09:37:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-12-01T09:37:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 30 k 2022-12-01T09:37:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 768 k 2022-12-01T09:37:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 139 k 2022-12-01T09:37:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-12-01T09:37:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-12-01T09:37:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-12-01T09:37:25.400 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-12-01T09:37:25.400 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-12-01T09:37:25.400 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-12-01T09:37:25.400 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 190 k 2022-12-01T09:37:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 370 k 2022-12-01T09:37:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 364 k 2022-12-01T09:37:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 107 k 2022-12-01T09:37:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 94 k 2022-12-01T09:37:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.5 M 2022-12-01T09:37:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 177 k 2022-12-01T09:37:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-12-01T09:37:25.402 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2022-12-01T09:37:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-12-01T09:37:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.6 M 2022-12-01T09:37:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.6 M 2022-12-01T09:37:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-12-01T09:37:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-12-01T09:37:25.404 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2022-12-01T09:37:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-12-01T09:37:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 24 M 2022-12-01T09:37:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 23 k 2022-12-01T09:37:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 2.3 M 2022-12-01T09:37:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 234 k 2022-12-01T09:37:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.9 M 2022-12-01T09:37:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 17 M 2022-12-01T09:37:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 14 k 2022-12-01T09:37:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 24 k 2022-12-01T09:37:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-12-01T09:37:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-12-01T09:37:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-12-01T09:37:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-12-01T09:37:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-12-01T09:37:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 137 k 2022-12-01T09:37:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-12-01T09:37:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-12-01T09:37:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-12-01T09:37:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-12-01T09:37:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-12-01T09:37:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-12-01T09:37:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 474 k 2022-12-01T09:37:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-12-01T09:37:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 6.5 M 2022-12-01T09:37:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-12-01T09:37:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-12-01T09:37:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-12-01T09:37:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-12-01T09:37:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-12-01T09:37:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-12-01T09:37:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-12-01T09:37:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-12-01T09:37:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-12-01T09:37:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-12-01T09:37:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-12-01T09:37:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-12-01T09:37:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-12-01T09:37:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-12-01T09:37:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-12-01T09:37:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-12-01T09:37:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 45 k 2022-12-01T09:37:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 109 k 2022-12-01T09:37:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-12-01T09:37:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-12-01T09:37:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-12-01T09:37:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-12-01T09:37:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-12-01T09:37:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-12-01T09:37:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-12-01T09:37:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-12-01T09:37:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-12-01T09:37:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-12-01T09:37:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-12-01T09:37:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-12-01T09:37:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-12-01T09:37:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-12-01T09:37:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-12-01T09:37:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-12-01T09:37:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-12-01T09:37:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-12-01T09:37:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-12-01T09:37:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-12-01T09:37:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-12-01T09:37:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-12-01T09:37:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-12-01T09:37:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-12-01T09:37:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-12-01T09:37:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-12-01T09:37:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-12-01T09:37:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-12-01T09:37:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-12-01T09:37:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-12-01T09:37:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-12-01T09:37:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-12-01T09:37:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-12-01T09:37:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-12-01T09:37:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-12-01T09:37:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-12-01T09:37:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-12-01T09:37:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-12-01T09:37:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-12-01T09:37:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-12-01T09:37:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-12-01T09:37:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-12-01T09:37:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-12-01T09:37:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-12-01T09:37:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-12-01T09:37:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-12-01T09:37:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-12-01T09:37:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-12-01T09:37:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-12-01T09:37:25.427 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling module streams: 2022-12-01T09:37:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql 10 2022-12-01T09:37:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:37:25.428 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2022-12-01T09:37:25.428 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2022-12-01T09:37:25.429 INFO:teuthology.orchestra.run.smithi002.stdout:Install 128 Packages 2022-12-01T09:37:25.429 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 5 Packages 2022-12-01T09:37:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:37:25.430 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 219 M 2022-12-01T09:37:25.431 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2022-12-01T09:37:25.439 INFO:teuthology.orchestra.run.smithi085.stdout:(4/133): ceph-mds-18.0.0-676.gf6922ca2.el8.x86_ 2.3 MB/s | 2.3 MB 00:00 2022-12-01T09:37:25.690 INFO:teuthology.orchestra.run.smithi002.stdout:(1/133): ceph-18.0.0-676.gf6922ca2.el8.x86_64.r 23 kB/s | 6.4 kB 00:00 2022-12-01T09:37:25.701 INFO:teuthology.orchestra.run.smithi085.stdout:(5/133): ceph-base-18.0.0-676.gf6922ca2.el8.x86 2.0 MB/s | 5.2 MB 00:02 2022-12-01T09:37:25.956 INFO:teuthology.orchestra.run.smithi085.stdout:(6/133): ceph-mgr-18.0.0-676.gf6922ca2.el8.x86_ 3.0 MB/s | 1.5 MB 00:00 2022-12-01T09:37:26.390 INFO:teuthology.orchestra.run.smithi085.stdout:(7/133): ceph-mon-18.0.0-676.gf6922ca2.el8.x86_ 5.6 MB/s | 3.9 MB 00:00 2022-12-01T09:37:26.541 INFO:teuthology.orchestra.run.smithi002.stdout:(2/133): ceph-fuse-18.0.0-676.gf6922ca2.el8.x86 994 kB/s | 845 kB 00:00 2022-12-01T09:37:26.857 INFO:teuthology.orchestra.run.smithi002.stdout:(3/133): ceph-immutable-object-cache-18.0.0-676 523 kB/s | 165 kB 00:00 2022-12-01T09:37:27.891 INFO:teuthology.orchestra.run.smithi002.stdout:(4/133): ceph-mds-18.0.0-676.gf6922ca2.el8.x86_ 2.2 MB/s | 2.3 MB 00:01 2022-12-01T09:37:27.966 INFO:teuthology.orchestra.run.smithi002.stdout:(5/133): ceph-base-18.0.0-676.gf6922ca2.el8.x86 2.0 MB/s | 5.2 MB 00:02 2022-12-01T09:37:28.550 INFO:teuthology.orchestra.run.smithi002.stdout:(6/133): ceph-mgr-18.0.0-676.gf6922ca2.el8.x86_ 2.3 MB/s | 1.5 MB 00:00 2022-12-01T09:37:29.050 INFO:teuthology.orchestra.run.smithi002.stdout:(7/133): ceph-mon-18.0.0-676.gf6922ca2.el8.x86_ 3.6 MB/s | 3.9 MB 00:01 2022-12-01T09:37:29.081 INFO:teuthology.orchestra.run.smithi085.stdout:(8/133): ceph-osd-18.0.0-676.gf6922ca2.el8.x86_ 5.4 MB/s | 17 MB 00:03 2022-12-01T09:37:29.173 INFO:teuthology.orchestra.run.smithi085.stdout:(9/133): ceph-selinux-18.0.0-676.gf6922ca2.el8. 266 kB/s | 24 kB 00:00 2022-12-01T09:37:30.590 INFO:teuthology.orchestra.run.smithi085.stdout:(10/133): ceph-common-18.0.0-676.gf6922ca2.el8. 3.3 MB/s | 24 MB 00:07 2022-12-01T09:37:30.815 INFO:teuthology.orchestra.run.smithi085.stdout:(11/133): cephfs-mirror-18.0.0-676.gf6922ca2.el 1.1 MB/s | 257 kB 00:00 2022-12-01T09:37:30.941 INFO:teuthology.orchestra.run.smithi085.stdout:(12/133): libcephfs-devel-18.0.0-676.gf6922ca2. 244 kB/s | 30 kB 00:00 2022-12-01T09:37:31.291 INFO:teuthology.orchestra.run.smithi085.stdout:(13/133): libcephfs2-18.0.0-676.gf6922ca2.el8.x 2.1 MB/s | 768 kB 00:00 2022-12-01T09:37:31.391 INFO:teuthology.orchestra.run.smithi085.stdout:(14/133): libcephsqlite-18.0.0-676.gf6922ca2.el 1.3 MB/s | 137 kB 00:00 2022-12-01T09:37:31.516 INFO:teuthology.orchestra.run.smithi085.stdout:(15/133): librados-devel-18.0.0-676.gf6922ca2.e 1.1 MB/s | 139 kB 00:00 2022-12-01T09:37:31.725 INFO:teuthology.orchestra.run.smithi002.stdout:(8/133): ceph-osd-18.0.0-676.gf6922ca2.el8.x86_ 5.3 MB/s | 17 MB 00:03 2022-12-01T09:37:31.817 INFO:teuthology.orchestra.run.smithi002.stdout:(9/133): ceph-selinux-18.0.0-676.gf6922ca2.el8. 266 kB/s | 24 kB 00:00 2022-12-01T09:37:31.891 INFO:teuthology.orchestra.run.smithi085.stdout:(16/133): libradosstriper1-18.0.0-676.gf6922ca2 1.2 MB/s | 474 kB 00:00 2022-12-01T09:37:32.408 INFO:teuthology.orchestra.run.smithi085.stdout:(17/133): ceph-radosgw-18.0.0-676.gf6922ca2.el8 3.0 MB/s | 18 MB 00:06 2022-12-01T09:37:32.492 INFO:teuthology.orchestra.run.smithi085.stdout:(18/133): python3-ceph-argparse-18.0.0-676.gf69 538 kB/s | 45 kB 00:00 2022-12-01T09:37:32.601 INFO:teuthology.orchestra.run.smithi085.stdout:(19/133): python3-ceph-common-18.0.0-676.gf6922 1.0 MB/s | 109 kB 00:00 2022-12-01T09:37:32.684 INFO:teuthology.orchestra.run.smithi085.stdout:(20/133): python3-cephfs-18.0.0-676.gf6922ca2.e 2.2 MB/s | 190 kB 00:00 2022-12-01T09:37:32.884 INFO:teuthology.orchestra.run.smithi085.stdout:(21/133): python3-rados-18.0.0-676.gf6922ca2.el 1.8 MB/s | 370 kB 00:00 2022-12-01T09:37:33.085 INFO:teuthology.orchestra.run.smithi085.stdout:(22/133): python3-rbd-18.0.0-676.gf6922ca2.el8. 1.8 MB/s | 364 kB 00:00 2022-12-01T09:37:33.235 INFO:teuthology.orchestra.run.smithi085.stdout:(23/133): python3-rgw-18.0.0-676.gf6922ca2.el8. 710 kB/s | 107 kB 00:00 2022-12-01T09:37:33.352 INFO:teuthology.orchestra.run.smithi085.stdout:(24/133): rbd-fuse-18.0.0-676.gf6922ca2.el8.x86 810 kB/s | 94 kB 00:00 2022-12-01T09:37:33.919 INFO:teuthology.orchestra.run.smithi085.stdout:(25/133): librgw2-18.0.0-676.gf6922ca2.el8.x86_ 3.2 MB/s | 6.5 MB 00:02 2022-12-01T09:37:34.036 INFO:teuthology.orchestra.run.smithi085.stdout:(26/133): rbd-nbd-18.0.0-676.gf6922ca2.el8.x86_ 1.5 MB/s | 177 kB 00:00 2022-12-01T09:37:34.136 INFO:teuthology.orchestra.run.smithi085.stdout:(27/133): ceph-grafana-dashboards-18.0.0-676.gf 229 kB/s | 23 kB 00:00 2022-12-01T09:37:34.278 INFO:teuthology.orchestra.run.smithi085.stdout:(28/133): ceph-mgr-cephadm-18.0.0-676.gf6922ca2 835 kB/s | 118 kB 00:00 2022-12-01T09:37:34.692 INFO:teuthology.orchestra.run.smithi002.stdout:(10/133): ceph-radosgw-18.0.0-676.gf6922ca2.el8 3.2 MB/s | 18 MB 00:05 2022-12-01T09:37:34.834 INFO:teuthology.orchestra.run.smithi002.stdout:(11/133): cephfs-mirror-18.0.0-676.gf6922ca2.el 1.8 MB/s | 257 kB 00:00 2022-12-01T09:37:34.884 INFO:teuthology.orchestra.run.smithi002.stdout:(12/133): libcephfs-devel-18.0.0-676.gf6922ca2. 611 kB/s | 30 kB 00:00 2022-12-01T09:37:34.904 INFO:teuthology.orchestra.run.smithi085.stdout:(29/133): rbd-mirror-18.0.0-676.gf6922ca2.el8.x 2.2 MB/s | 3.5 MB 00:01 2022-12-01T09:37:34.954 INFO:teuthology.orchestra.run.smithi085.stdout:(30/133): ceph-mgr-dashboard-18.0.0-676.gf6922c 2.4 MB/s | 1.7 MB 00:00 2022-12-01T09:37:35.126 INFO:teuthology.orchestra.run.smithi002.stdout:(13/133): libcephfs2-18.0.0-676.gf6922ca2.el8.x 3.1 MB/s | 768 kB 00:00 2022-12-01T09:37:35.179 INFO:teuthology.orchestra.run.smithi085.stdout:(31/133): ceph-mgr-modules-core-18.0.0-676.gf69 1.0 MB/s | 234 kB 00:00 2022-12-01T09:37:35.226 INFO:teuthology.orchestra.run.smithi002.stdout:(14/133): libcephsqlite-18.0.0-676.gf6922ca2.el 1.3 MB/s | 137 kB 00:00 2022-12-01T09:37:35.246 INFO:teuthology.orchestra.run.smithi085.stdout:(32/133): ceph-mgr-rook-18.0.0-676.gf6922ca2.el 705 kB/s | 47 kB 00:00 2022-12-01T09:37:35.313 INFO:teuthology.orchestra.run.smithi085.stdout:(33/133): ceph-prometheus-alerts-18.0.0-676.gf6 216 kB/s | 14 kB 00:00 2022-12-01T09:37:35.327 INFO:teuthology.orchestra.run.smithi002.stdout:(15/133): librados-devel-18.0.0-676.gf6922ca2.e 1.4 MB/s | 139 kB 00:00 2022-12-01T09:37:35.463 INFO:teuthology.orchestra.run.smithi085.stdout:(34/133): ceph-volume-18.0.0-676.gf6922ca2.el8. 1.7 MB/s | 263 kB 00:00 2022-12-01T09:37:35.647 INFO:teuthology.orchestra.run.smithi085.stdout:(35/133): cephadm-18.0.0-676.gf6922ca2.el8.noar 443 kB/s | 81 kB 00:00 2022-12-01T09:37:35.744 INFO:teuthology.orchestra.run.smithi002.stdout:(16/133): libradosstriper1-18.0.0-676.gf6922ca2 1.1 MB/s | 474 kB 00:00 2022-12-01T09:37:35.755 INFO:teuthology.orchestra.run.smithi085.stdout:(36/133): cephfs-top-18.0.0-676.gf6922ca2.el8.n 142 kB/s | 15 kB 00:00 2022-12-01T09:37:36.047 INFO:teuthology.orchestra.run.smithi085.stdout:(37/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2022-12-01T09:37:36.114 INFO:teuthology.orchestra.run.smithi085.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 884 kB/s | 59 kB 00:00 2022-12-01T09:37:36.181 INFO:teuthology.orchestra.run.smithi085.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2022-12-01T09:37:36.565 INFO:teuthology.orchestra.run.smithi085.stdout:(40/133): python3-asyncssh-2.7.0-2.el8.noarch.r 1.2 MB/s | 476 kB 00:00 2022-12-01T09:37:36.640 INFO:teuthology.orchestra.run.smithi085.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 584 kB/s | 44 kB 00:00 2022-12-01T09:37:36.710 INFO:teuthology.orchestra.run.smithi002.stdout:(17/133): ceph-common-18.0.0-676.gf6922ca2.el8. 2.2 MB/s | 24 MB 00:11 2022-12-01T09:37:36.757 INFO:teuthology.orchestra.run.smithi085.stdout:(42/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2022-12-01T09:37:36.828 INFO:teuthology.orchestra.run.smithi002.stdout:(18/133): python3-ceph-argparse-18.0.0-676.gf69 550 kB/s | 45 kB 00:00 2022-12-01T09:37:36.833 INFO:teuthology.orchestra.run.smithi085.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 441 kB/s | 33 kB 00:00 2022-12-01T09:37:36.916 INFO:teuthology.orchestra.run.smithi085.stdout:(44/133): python3-certifi-2018.10.15-7.el8.noar 196 kB/s | 16 kB 00:00 2022-12-01T09:37:36.944 INFO:teuthology.orchestra.run.smithi002.stdout:(19/133): python3-ceph-common-18.0.0-676.gf6922 937 kB/s | 109 kB 00:00 2022-12-01T09:37:36.991 INFO:teuthology.orchestra.run.smithi085.stdout:(45/133): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2022-12-01T09:37:37.102 INFO:teuthology.orchestra.run.smithi002.stdout:(20/133): python3-cephfs-18.0.0-676.gf6922ca2.e 1.2 MB/s | 190 kB 00:00 2022-12-01T09:37:37.125 INFO:teuthology.orchestra.run.smithi085.stdout:(46/133): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2022-12-01T09:37:37.209 INFO:teuthology.orchestra.run.smithi085.stdout:(47/133): python3-google-auth-1.1.1-10.el8.noar 975 kB/s | 81 kB 00:00 2022-12-01T09:37:37.284 INFO:teuthology.orchestra.run.smithi085.stdout:(48/133): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2022-12-01T09:37:37.287 INFO:teuthology.orchestra.run.smithi002.stdout:(21/133): python3-rados-18.0.0-676.gf6922ca2.el 2.0 MB/s | 370 kB 00:00 2022-12-01T09:37:37.368 INFO:teuthology.orchestra.run.smithi085.stdout:(49/133): python3-jaraco-functools-2.0-4.el8.no 211 kB/s | 18 kB 00:00 2022-12-01T09:37:37.479 INFO:teuthology.orchestra.run.smithi002.stdout:(22/133): python3-rbd-18.0.0-676.gf6922ca2.el8. 1.9 MB/s | 364 kB 00:00 2022-12-01T09:37:37.510 INFO:teuthology.orchestra.run.smithi085.stdout:(50/133): ceph-mgr-diskprediction-local-18.0.0- 2.8 MB/s | 7.4 MB 00:02 2022-12-01T09:37:37.554 INFO:teuthology.orchestra.run.smithi002.stdout:(23/133): python3-rgw-18.0.0-676.gf6922ca2.el8. 1.4 MB/s | 107 kB 00:00 2022-12-01T09:37:37.638 INFO:teuthology.orchestra.run.smithi002.stdout:(24/133): rbd-fuse-18.0.0-676.gf6922ca2.el8.x86 1.1 MB/s | 94 kB 00:00 2022-12-01T09:37:37.785 INFO:teuthology.orchestra.run.smithi085.stdout:(51/133): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2022-12-01T09:37:37.810 INFO:teuthology.orchestra.run.smithi085.stdout:(52/133): python3-logutils-0.3.5-11.el8.noarch. 162 kB/s | 49 kB 00:00 2022-12-01T09:37:37.869 INFO:teuthology.orchestra.run.smithi085.stdout:(53/133): python3-more-itertools-7.2.0-3.el8.no 705 kB/s | 59 kB 00:00 2022-12-01T09:37:37.961 INFO:teuthology.orchestra.run.smithi085.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 401 kB/s | 60 kB 00:00 2022-12-01T09:37:38.003 INFO:teuthology.orchestra.run.smithi085.stdout:(55/133): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2022-12-01T09:37:38.037 INFO:teuthology.orchestra.run.smithi085.stdout:(56/133): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2022-12-01T09:37:38.070 INFO:teuthology.orchestra.run.smithi085.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2022-12-01T09:37:38.129 INFO:teuthology.orchestra.run.smithi002.stdout:(25/133): librgw2-18.0.0-676.gf6922ca2.el8.x86_ 2.7 MB/s | 6.5 MB 00:02 2022-12-01T09:37:38.145 INFO:teuthology.orchestra.run.smithi085.stdout:(58/133): python3-rsa-4.8-1.el8.noarch.rpm 819 kB/s | 61 kB 00:00 2022-12-01T09:37:38.199 INFO:teuthology.orchestra.run.smithi085.stdout:(59/133): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2022-12-01T09:37:38.221 INFO:teuthology.orchestra.run.smithi085.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2022-12-01T09:37:38.255 INFO:teuthology.orchestra.run.smithi002.stdout:(26/133): rbd-nbd-18.0.0-676.gf6922ca2.el8.x86_ 1.4 MB/s | 177 kB 00:00 2022-12-01T09:37:38.271 INFO:teuthology.orchestra.run.smithi085.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 335 kB/s | 24 kB 00:00 2022-12-01T09:37:38.288 INFO:teuthology.orchestra.run.smithi085.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 437 kB/s | 29 kB 00:00 2022-12-01T09:37:38.330 INFO:teuthology.orchestra.run.smithi002.stdout:(27/133): ceph-grafana-dashboards-18.0.0-676.gf 306 kB/s | 23 kB 00:00 2022-12-01T09:37:38.347 INFO:teuthology.orchestra.run.smithi085.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2022-12-01T09:37:38.380 INFO:teuthology.orchestra.run.smithi085.stdout:(64/133): python3-waitress-1.2.1-2.el8.1.noarch 2.6 MB/s | 241 kB 00:00 2022-12-01T09:37:38.413 INFO:teuthology.orchestra.run.smithi002.stdout:(28/133): ceph-mgr-cephadm-18.0.0-676.gf6922ca2 1.4 MB/s | 118 kB 00:00 2022-12-01T09:37:38.464 INFO:teuthology.orchestra.run.smithi085.stdout:(65/133): python3-websocket-client-0.56.0-5.el8 734 kB/s | 61 kB 00:00 2022-12-01T09:37:38.498 INFO:teuthology.orchestra.run.smithi085.stdout:(66/133): python3-webob-1.8.5-1.el8.1.noarch.rp 1.6 MB/s | 251 kB 00:00 2022-12-01T09:37:38.548 INFO:teuthology.orchestra.run.smithi085.stdout:(67/133): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2022-12-01T09:37:38.581 INFO:teuthology.orchestra.run.smithi085.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 275 kB/s | 23 kB 00:00 2022-12-01T09:37:38.598 INFO:teuthology.orchestra.run.smithi085.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2022-12-01T09:37:38.657 INFO:teuthology.orchestra.run.smithi085.stdout:(70/133): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2022-12-01T09:37:38.699 INFO:teuthology.orchestra.run.smithi085.stdout:(71/133): fuse-2.9.7-15.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2022-12-01T09:37:38.741 INFO:teuthology.orchestra.run.smithi085.stdout:(72/133): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2022-12-01T09:37:38.784 INFO:teuthology.orchestra.run.smithi085.stdout:(73/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2022-12-01T09:37:38.825 INFO:teuthology.orchestra.run.smithi085.stdout:(74/133): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2022-12-01T09:37:38.867 INFO:teuthology.orchestra.run.smithi085.stdout:(75/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 430 kB/s | 18 kB 00:00 2022-12-01T09:37:38.901 INFO:teuthology.orchestra.run.smithi085.stdout:(76/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2022-12-01T09:37:38.951 INFO:teuthology.orchestra.run.smithi085.stdout:(77/133): python3-cryptography-3.2.1-5.el8.x86_ 11 MB/s | 559 kB 00:00 2022-12-01T09:37:38.993 INFO:teuthology.orchestra.run.smithi085.stdout:(78/133): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2022-12-01T09:37:39.027 INFO:teuthology.orchestra.run.smithi085.stdout:(79/133): python3-pycparser-2.14-14.el8.noarch. 3.2 MB/s | 109 kB 00:00 2022-12-01T09:37:39.047 INFO:teuthology.orchestra.run.smithi002.stdout:(29/133): ceph-mgr-dashboard-18.0.0-676.gf6922c 2.6 MB/s | 1.7 MB 00:00 2022-12-01T09:37:39.094 INFO:teuthology.orchestra.run.smithi085.stdout:(80/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2022-12-01T09:37:39.106 INFO:teuthology.orchestra.run.smithi002.stdout:(30/133): rbd-mirror-18.0.0-676.gf6922ca2.el8.x 2.4 MB/s | 3.5 MB 00:01 2022-12-01T09:37:39.127 INFO:teuthology.orchestra.run.smithi085.stdout:(81/133): thrift-0.13.0-2.el8.x86_64.rpm 3.0 MB/s | 1.7 MB 00:00 2022-12-01T09:37:39.161 INFO:teuthology.orchestra.run.smithi085.stdout:(82/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2022-12-01T09:37:39.195 INFO:teuthology.orchestra.run.smithi085.stdout:(83/133): python3-jwt-1.6.1-2.el8.noarch.rpm 646 kB/s | 43 kB 00:00 2022-12-01T09:37:39.220 INFO:teuthology.orchestra.run.smithi085.stdout:(84/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2022-12-01T09:37:39.263 INFO:teuthology.orchestra.run.smithi085.stdout:(85/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2022-12-01T09:37:39.287 INFO:teuthology.orchestra.run.smithi085.stdout:(86/133): elfutils-libelf-devel-0.186-1.el8.x86 651 kB/s | 60 kB 00:00 2022-12-01T09:37:39.321 INFO:teuthology.orchestra.run.smithi085.stdout:(87/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2022-12-01T09:37:39.346 INFO:teuthology.orchestra.run.smithi085.stdout:(88/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.6 MB/s | 98 kB 00:00 2022-12-01T09:37:39.388 INFO:teuthology.orchestra.run.smithi085.stdout:(89/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.4 MB/s | 229 kB 00:00 2022-12-01T09:37:39.413 INFO:teuthology.orchestra.run.smithi085.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 647 kB/s | 43 kB 00:00 2022-12-01T09:37:39.455 INFO:teuthology.orchestra.run.smithi085.stdout:(91/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2022-12-01T09:37:39.500 INFO:teuthology.orchestra.run.smithi085.stdout:(92/133): libcom_err-devel-1.45.6-4.el8.x86_64. 444 kB/s | 39 kB 00:00 2022-12-01T09:37:39.547 INFO:teuthology.orchestra.run.smithi085.stdout:(93/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.0 MB/s | 48 kB 00:00 2022-12-01T09:37:39.606 INFO:teuthology.orchestra.run.smithi085.stdout:(94/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2022-12-01T09:37:39.607 INFO:teuthology.orchestra.run.smithi002.stdout:(31/133): ceph-test-18.0.0-676.gf6922ca2.el8.x8 7.0 MB/s | 55 MB 00:07 2022-12-01T09:37:39.639 INFO:teuthology.orchestra.run.smithi085.stdout:(95/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.9 MB/s | 644 kB 00:00 2022-12-01T09:37:39.640 INFO:teuthology.orchestra.run.smithi002.stdout:(32/133): ceph-mgr-modules-core-18.0.0-676.gf69 438 kB/s | 234 kB 00:00 2022-12-01T09:37:39.673 INFO:teuthology.orchestra.run.smithi085.stdout:(96/133): NetworkManager-initscripts-updown-1.3 2.0 MB/s | 138 kB 00:00 2022-12-01T09:37:39.682 INFO:teuthology.orchestra.run.smithi002.stdout:(33/133): ceph-mgr-rook-18.0.0-676.gf6922ca2.el 625 kB/s | 47 kB 00:00 2022-12-01T09:37:39.715 INFO:teuthology.orchestra.run.smithi085.stdout:(97/133): postgresql-contrib-10.21-2.module+el8 11 MB/s | 808 kB 00:00 2022-12-01T09:37:39.716 INFO:teuthology.orchestra.run.smithi002.stdout:(34/133): ceph-prometheus-alerts-18.0.0-676.gf6 192 kB/s | 14 kB 00:00 2022-12-01T09:37:39.749 INFO:teuthology.orchestra.run.smithi085.stdout:(98/133): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2022-12-01T09:37:39.816 INFO:teuthology.orchestra.run.smithi085.stdout:(99/133): postgresql-10.21-2.module+el8.6.0+153 15 MB/s | 1.5 MB 00:00 2022-12-01T09:37:39.824 INFO:teuthology.orchestra.run.smithi002.stdout:(35/133): ceph-volume-18.0.0-676.gf6922ca2.el8. 1.8 MB/s | 263 kB 00:00 2022-12-01T09:37:39.849 INFO:teuthology.orchestra.run.smithi002.stdout:(36/133): cephadm-18.0.0-676.gf6922ca2.el8.noar 608 kB/s | 81 kB 00:00 2022-12-01T09:37:39.874 INFO:teuthology.orchestra.run.smithi085.stdout:(100/133): python3-numpy-1.14.3-10.el8.x86_64.r 30 MB/s | 3.7 MB 00:00 2022-12-01T09:37:39.900 INFO:teuthology.orchestra.run.smithi002.stdout:(37/133): cephfs-top-18.0.0-676.gf6922ca2.el8.n 205 kB/s | 15 kB 00:00 2022-12-01T09:37:39.966 INFO:teuthology.orchestra.run.smithi085.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 4.9 MB/s | 457 kB 00:00 2022-12-01T09:37:40.041 INFO:teuthology.orchestra.run.smithi085.stdout:(102/133): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2022-12-01T09:37:40.066 INFO:teuthology.orchestra.run.smithi085.stdout:(103/133): python3-markupsafe-0.23-19.el8.x86_6 388 kB/s | 39 kB 00:00 2022-12-01T09:37:40.101 INFO:teuthology.orchestra.run.smithi085.stdout:(104/133): python3-prettytable-0.7.2-14.el8.noa 752 kB/s | 44 kB 00:00 2022-12-01T09:37:40.126 INFO:teuthology.orchestra.run.smithi085.stdout:(105/133): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2022-12-01T09:37:40.168 INFO:teuthology.orchestra.run.smithi085.stdout:(106/133): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2022-12-01T09:37:40.175 INFO:teuthology.orchestra.run.smithi002.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 214 kB/s | 59 kB 00:00 2022-12-01T09:37:40.210 INFO:teuthology.orchestra.run.smithi085.stdout:(107/133): libtool-2.4.6-25.el8.x86_64.rpm 8.3 MB/s | 709 kB 00:00 2022-12-01T09:37:40.233 INFO:teuthology.orchestra.run.smithi002.stdout:(39/133): gperftools-libs-2.7-9.el8.x86_64.rpm 799 kB/s | 306 kB 00:00 2022-12-01T09:37:40.243 INFO:teuthology.orchestra.run.smithi085.stdout:(108/133): python3-numpy-f2py-1.14.3-10.el8.x86 2.9 MB/s | 225 kB 00:00 2022-12-01T09:37:40.286 INFO:teuthology.orchestra.run.smithi085.stdout:(109/133): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2022-12-01T09:37:40.325 INFO:teuthology.orchestra.run.smithi002.stdout:(40/133): libunwind-1.3.1-3.el8.x86_64.rpm 500 kB/s | 75 kB 00:00 2022-12-01T09:37:40.369 INFO:teuthology.orchestra.run.smithi085.stdout:(110/133): openblas-threads-0.3.15-3.el8.x86_64 31 MB/s | 4.9 MB 00:00 2022-12-01T09:37:40.409 INFO:teuthology.orchestra.run.smithi002.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 524 kB/s | 44 kB 00:00 2022-12-01T09:37:40.419 INFO:teuthology.orchestra.run.smithi085.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2022-12-01T09:37:40.444 INFO:teuthology.orchestra.run.smithi002.stdout:(42/133): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2022-12-01T09:37:40.495 INFO:teuthology.orchestra.run.smithi085.stdout:(112/133): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2022-12-01T09:37:40.520 INFO:teuthology.orchestra.run.smithi085.stdout:(113/133): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2022-12-01T09:37:40.526 INFO:teuthology.orchestra.run.smithi002.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 396 kB/s | 33 kB 00:00 2022-12-01T09:37:40.560 INFO:teuthology.orchestra.run.smithi002.stdout:(44/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2022-12-01T09:37:40.562 INFO:teuthology.orchestra.run.smithi085.stdout:(114/133): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2022-12-01T09:37:40.604 INFO:teuthology.orchestra.run.smithi085.stdout:(115/133): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2022-12-01T09:37:40.660 INFO:teuthology.orchestra.run.smithi002.stdout:(45/133): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2022-12-01T09:37:40.687 INFO:teuthology.orchestra.run.smithi085.stdout:(116/133): openblas-0.3.15-3.el8.x86_64.rpm 28 MB/s | 4.7 MB 00:00 2022-12-01T09:37:40.737 INFO:teuthology.orchestra.run.smithi085.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-12-01T09:37:40.763 INFO:teuthology.orchestra.run.smithi085.stdout:(118/133): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2022-12-01T09:37:40.805 INFO:teuthology.orchestra.run.smithi085.stdout:(119/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.0 MB/s | 126 kB 00:00 2022-12-01T09:37:40.810 INFO:teuthology.orchestra.run.smithi002.stdout:(46/133): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2022-12-01T09:37:40.855 INFO:teuthology.orchestra.run.smithi085.stdout:(120/133): python3-jsonpointer-1.10-11.el8.noar 402 kB/s | 20 kB 00:00 2022-12-01T09:37:40.902 INFO:teuthology.orchestra.run.smithi002.stdout:(47/133): python3-google-auth-1.1.1-10.el8.noar 888 kB/s | 81 kB 00:00 2022-12-01T09:37:41.002 INFO:teuthology.orchestra.run.smithi002.stdout:(48/133): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2022-12-01T09:37:41.005 INFO:teuthology.orchestra.run.smithi085.stdout:(121/133): python3-scipy-1.0.0-21.module+el8.5. 34 MB/s | 14 MB 00:00 2022-12-01T09:37:41.030 INFO:teuthology.orchestra.run.smithi085.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2022-12-01T09:37:41.061 INFO:teuthology.orchestra.run.smithi002.stdout:(49/133): python3-certifi-2018.10.15-7.el8.noar 30 kB/s | 16 kB 00:00 2022-12-01T09:37:41.073 INFO:teuthology.orchestra.run.smithi085.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2022-12-01T09:37:41.086 INFO:teuthology.orchestra.run.smithi002.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2022-12-01T09:37:41.098 INFO:teuthology.orchestra.run.smithi085.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2022-12-01T09:37:41.131 INFO:teuthology.orchestra.run.smithi085.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 454 kB/s | 27 kB 00:00 2022-12-01T09:37:41.157 INFO:teuthology.orchestra.run.smithi085.stdout:(126/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-12-01T09:37:41.178 INFO:teuthology.orchestra.run.smithi002.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 531 kB/s | 49 kB 00:00 2022-12-01T09:37:41.190 INFO:teuthology.orchestra.run.smithi085.stdout:(127/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.8 MB/s | 110 kB 00:00 2022-12-01T09:37:41.262 INFO:teuthology.orchestra.run.smithi002.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 706 kB/s | 59 kB 00:00 2022-12-01T09:37:41.345 INFO:teuthology.orchestra.run.smithi002.stdout:(53/133): python3-natsort-7.1.1-2.el8.noarch.rp 723 kB/s | 60 kB 00:00 2022-12-01T09:37:41.504 INFO:teuthology.orchestra.run.smithi002.stdout:(54/133): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2022-12-01T09:37:41.596 INFO:teuthology.orchestra.run.smithi002.stdout:(55/133): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2022-12-01T09:37:41.696 INFO:teuthology.orchestra.run.smithi002.stdout:(56/133): python3-repoze-lru-0.7-6.el8.noarch.r 337 kB/s | 34 kB 00:00 2022-12-01T09:37:41.738 INFO:teuthology.orchestra.run.smithi002.stdout:(57/133): python3-kubernetes-11.0.0-6.el8.noarc 1.7 MB/s | 1.2 MB 00:00 2022-12-01T09:37:41.780 INFO:teuthology.orchestra.run.smithi002.stdout:(58/133): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2022-12-01T09:37:41.828 INFO:teuthology.orchestra.run.smithi002.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 736 kB/s | 61 kB 00:00 2022-12-01T09:37:41.864 INFO:teuthology.orchestra.run.smithi002.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 223 kB/s | 19 kB 00:00 2022-12-01T09:37:41.931 INFO:teuthology.orchestra.run.smithi002.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2022-12-01T09:37:41.965 INFO:teuthology.orchestra.run.smithi002.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2022-12-01T09:37:42.073 INFO:teuthology.orchestra.run.smithi002.stdout:(63/133): ceph-mgr-diskprediction-local-18.0.0- 2.4 MB/s | 7.4 MB 00:03 2022-12-01T09:37:42.099 INFO:teuthology.orchestra.run.smithi002.stdout:(64/133): python3-trustme-0.6.0-4.el8.noarch.rp 164 kB/s | 27 kB 00:00 2022-12-01T09:37:42.124 INFO:teuthology.orchestra.run.smithi002.stdout:(65/133): python3-waitress-1.2.1-2.el8.1.noarch 1.5 MB/s | 241 kB 00:00 2022-12-01T09:37:42.183 INFO:teuthology.orchestra.run.smithi002.stdout:(66/133): python3-websocket-client-0.56.0-5.el8 732 kB/s | 61 kB 00:00 2022-12-01T09:37:42.199 INFO:teuthology.orchestra.run.smithi002.stdout:(67/133): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2022-12-01T09:37:42.354 INFO:teuthology.orchestra.run.smithi002.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 134 kB/s | 23 kB 00:00 2022-12-01T09:37:42.600 INFO:teuthology.orchestra.run.smithi002.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 285 kB/s | 70 kB 00:00 2022-12-01T09:37:42.625 INFO:teuthology.orchestra.run.smithi002.stdout:(70/133): python3-webob-1.8.5-1.el8.1.noarch.rp 456 kB/s | 251 kB 00:00 2022-12-01T09:37:42.657 INFO:teuthology.orchestra.run.smithi085.stdout:(128/133): librados2-18.0.0-676.gf6922ca2.el8.x 2.4 MB/s | 3.6 MB 00:01 2022-12-01T09:37:42.692 INFO:teuthology.orchestra.run.smithi002.stdout:(71/133): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2022-12-01T09:37:42.708 INFO:teuthology.orchestra.run.smithi085.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 14 MB/s | 709 kB 00:00 2022-12-01T09:37:42.726 INFO:teuthology.orchestra.run.smithi002.stdout:(72/133): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2022-12-01T09:37:42.760 INFO:teuthology.orchestra.run.smithi002.stdout:(73/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2022-12-01T09:37:42.775 INFO:teuthology.orchestra.run.smithi085.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 22 MB/s | 1.5 MB 00:00 2022-12-01T09:37:42.784 INFO:teuthology.orchestra.run.smithi002.stdout:(74/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.8 MB/s | 220 kB 00:00 2022-12-01T09:37:42.827 INFO:teuthology.orchestra.run.smithi002.stdout:(75/133): libselinux-devel-2.9-5.el8.x86_64.rpm 2.9 MB/s | 200 kB 00:00 2022-12-01T09:37:42.833 INFO:teuthology.orchestra.run.smithi085.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2022-12-01T09:37:42.852 INFO:teuthology.orchestra.run.smithi002.stdout:(76/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 273 kB/s | 18 kB 00:00 2022-12-01T09:37:42.886 INFO:teuthology.orchestra.run.smithi002.stdout:(77/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 801 kB/s | 47 kB 00:00 2022-12-01T09:37:42.927 INFO:teuthology.orchestra.run.smithi002.stdout:(78/133): thrift-0.13.0-2.el8.x86_64.rpm 2.4 MB/s | 1.7 MB 00:00 2022-12-01T09:37:42.942 INFO:teuthology.orchestra.run.smithi085.stdout:(132/133): librbd1-18.0.0-676.gf6922ca2.el8.x86 2.1 MB/s | 3.6 MB 00:01 2022-12-01T09:37:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:(79/133): python3-cryptography-3.2.1-5.el8.x86_ 5.0 MB/s | 559 kB 00:00 2022-12-01T09:37:42.986 INFO:teuthology.orchestra.run.smithi002.stdout:(80/133): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2022-12-01T09:37:43.020 INFO:teuthology.orchestra.run.smithi002.stdout:(81/133): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2022-12-01T09:37:43.053 INFO:teuthology.orchestra.run.smithi002.stdout:(82/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.0 MB/s | 560 kB 00:00 2022-12-01T09:37:43.079 INFO:teuthology.orchestra.run.smithi002.stdout:(83/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.9 MB/s | 171 kB 00:00 2022-12-01T09:37:43.096 INFO:teuthology.orchestra.run.smithi002.stdout:(84/133): python3-jwt-1.6.1-2.el8.noarch.rpm 573 kB/s | 43 kB 00:00 2022-12-01T09:37:43.121 INFO:teuthology.orchestra.run.smithi002.stdout:(85/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 876 kB/s | 58 kB 00:00 2022-12-01T09:37:43.138 INFO:teuthology.orchestra.run.smithi002.stdout:(86/133): elfutils-libelf-devel-0.186-1.el8.x86 1.0 MB/s | 60 kB 00:00 2022-12-01T09:37:43.163 INFO:teuthology.orchestra.run.smithi002.stdout:(87/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2022-12-01T09:37:43.189 INFO:teuthology.orchestra.run.smithi002.stdout:(88/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 279 kB/s | 19 kB 00:00 2022-12-01T09:37:43.205 INFO:teuthology.orchestra.run.smithi002.stdout:(89/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.5 MB/s | 98 kB 00:00 2022-12-01T09:37:43.231 INFO:teuthology.orchestra.run.smithi002.stdout:(90/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.4 MB/s | 229 kB 00:00 2022-12-01T09:37:43.264 INFO:teuthology.orchestra.run.smithi002.stdout:(91/133): python3-requests-oauthlib-1.0.0-1.el8 568 kB/s | 43 kB 00:00 2022-12-01T09:37:43.290 INFO:teuthology.orchestra.run.smithi002.stdout:(92/133): libcom_err-devel-1.45.6-4.el8.x86_64. 660 kB/s | 39 kB 00:00 2022-12-01T09:37:43.315 INFO:teuthology.orchestra.run.smithi002.stdout:(93/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 669 kB/s | 72 kB 00:00 2022-12-01T09:37:43.357 INFO:teuthology.orchestra.run.smithi002.stdout:(94/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 721 kB/s | 48 kB 00:00 2022-12-01T09:37:43.390 INFO:teuthology.orchestra.run.smithi002.stdout:(95/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 8.5 MB/s | 644 kB 00:00 2022-12-01T09:37:43.440 INFO:teuthology.orchestra.run.smithi002.stdout:(96/133): NetworkManager-initscripts-updown-1.3 1.6 MB/s | 138 kB 00:00 2022-12-01T09:37:43.500 INFO:teuthology.orchestra.run.smithi002.stdout:(97/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2022-12-01T09:37:43.533 INFO:teuthology.orchestra.run.smithi002.stdout:(98/133): bison-3.0.4-10.el8.x86_64.rpm 7.4 MB/s | 688 kB 00:00 2022-12-01T09:37:43.567 INFO:teuthology.orchestra.run.smithi002.stdout:(99/133): postgresql-contrib-10.21-2.module+el8 4.5 MB/s | 808 kB 00:00 2022-12-01T09:37:43.617 INFO:teuthology.orchestra.run.smithi002.stdout:(100/133): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2022-12-01T09:37:43.659 INFO:teuthology.orchestra.run.smithi002.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2022-12-01T09:37:43.701 INFO:teuthology.orchestra.run.smithi002.stdout:(102/133): python3-markupsafe-0.23-19.el8.x86_6 940 kB/s | 39 kB 00:00 2022-12-01T09:37:43.734 INFO:teuthology.orchestra.run.smithi085.stdout:(133/133): ceph-test-18.0.0-676.gf6922ca2.el8.x 3.8 MB/s | 55 MB 00:14 2022-12-01T09:37:43.737 INFO:teuthology.orchestra.run.smithi085.stdout:-------------------------------------------------------------------------------- 2022-12-01T09:37:43.737 INFO:teuthology.orchestra.run.smithi085.stdout:Total 11 MB/s | 219 MB 00:20 2022-12-01T09:37:43.767 INFO:teuthology.orchestra.run.smithi002.stdout:(103/133): python3-numpy-1.14.3-10.el8.x86_64.r 16 MB/s | 3.7 MB 00:00 2022-12-01T09:37:43.801 INFO:teuthology.orchestra.run.smithi002.stdout:(104/133): python3-prettytable-0.7.2-14.el8.noa 439 kB/s | 44 kB 00:00 2022-12-01T09:37:43.877 INFO:teuthology.orchestra.run.smithi002.stdout:(105/133): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2022-12-01T09:37:43.902 INFO:teuthology.orchestra.run.smithi002.stdout:(106/133): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2022-12-01T09:37:43.935 INFO:teuthology.orchestra.run.smithi002.stdout:(107/133): python3-jinja2-2.10.1-3.el8.noarch.r 4.0 MB/s | 538 kB 00:00 2022-12-01T09:37:43.969 INFO:teuthology.orchestra.run.smithi002.stdout:(108/133): libtool-2.4.6-25.el8.x86_64.rpm 7.6 MB/s | 709 kB 00:00 2022-12-01T09:37:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:(109/133): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2022-12-01T09:37:44.028 INFO:teuthology.orchestra.run.smithi002.stdout:(110/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.7 MB/s | 157 kB 00:00 2022-12-01T09:37:44.070 INFO:teuthology.orchestra.run.smithi002.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2022-12-01T09:37:44.112 INFO:teuthology.orchestra.run.smithi002.stdout:(112/133): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2022-12-01T09:37:44.209 INFO:teuthology.orchestra.run.smithi002.stdout:(113/133): openblas-threads-0.3.15-3.el8.x86_64 18 MB/s | 4.9 MB 00:00 2022-12-01T09:37:44.245 INFO:teuthology.orchestra.run.smithi002.stdout:(114/133): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2022-12-01T09:37:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:(115/133): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2022-12-01T09:37:44.346 INFO:teuthology.orchestra.run.smithi002.stdout:(116/133): jq-1.6-3.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2022-12-01T09:37:44.396 INFO:teuthology.orchestra.run.smithi002.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-12-01T09:37:44.472 INFO:teuthology.orchestra.run.smithi002.stdout:(118/133): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2022-12-01T09:37:44.488 INFO:teuthology.orchestra.run.smithi002.stdout:(119/133): python3-pytz-2017.2-9.el8.noarch.rpm 590 kB/s | 54 kB 00:00 2022-12-01T09:37:44.530 INFO:teuthology.orchestra.run.smithi002.stdout:(120/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.1 MB/s | 126 kB 00:00 2022-12-01T09:37:44.547 INFO:teuthology.orchestra.run.smithi002.stdout:(121/133): python3-jsonpointer-1.10-11.el8.noar 342 kB/s | 20 kB 00:00 2022-12-01T09:37:44.581 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction check 2022-12-01T09:37:44.581 INFO:teuthology.orchestra.run.smithi002.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2022-12-01T09:37:44.623 INFO:teuthology.orchestra.run.smithi002.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2022-12-01T09:37:44.665 INFO:teuthology.orchestra.run.smithi002.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2022-12-01T09:37:44.690 INFO:teuthology.orchestra.run.smithi002.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 403 kB/s | 27 kB 00:00 2022-12-01T09:37:44.724 INFO:teuthology.orchestra.run.smithi002.stdout:(126/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-12-01T09:37:44.739 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction check succeeded. 2022-12-01T09:37:44.739 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction test 2022-12-01T09:37:44.749 INFO:teuthology.orchestra.run.smithi002.stdout:(127/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.8 MB/s | 110 kB 00:00 2022-12-01T09:37:44.908 INFO:teuthology.orchestra.run.smithi002.stdout:(128/133): python3-scipy-1.0.0-21.module+el8.5. 23 MB/s | 14 MB 00:00 2022-12-01T09:37:44.950 INFO:teuthology.orchestra.run.smithi002.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 17 MB/s | 709 kB 00:00 2022-12-01T09:37:45.009 INFO:teuthology.orchestra.run.smithi002.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 1.5 MB 00:00 2022-12-01T09:37:45.075 INFO:teuthology.orchestra.run.smithi002.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2022-12-01T09:37:45.667 INFO:teuthology.orchestra.run.smithi002.stdout:(132/133): librados2-18.0.0-676.gf6922ca2.el8.x 3.8 MB/s | 3.6 MB 00:00 2022-12-01T09:37:45.912 INFO:teuthology.orchestra.run.smithi085.stdout:Transaction test succeeded. 2022-12-01T09:37:45.915 INFO:teuthology.orchestra.run.smithi085.stdout:Running transaction 2022-12-01T09:37:46.592 INFO:teuthology.orchestra.run.smithi002.stdout:(133/133): librbd1-18.0.0-676.gf6922ca2.el8.x86 2.0 MB/s | 3.6 MB 00:01 2022-12-01T09:37:46.595 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2022-12-01T09:37:46.596 INFO:teuthology.orchestra.run.smithi002.stdout:Total 10 MB/s | 219 MB 00:21 2022-12-01T09:37:47.524 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2022-12-01T09:37:47.702 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2022-12-01T09:37:47.702 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2022-12-01T09:37:48.936 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2022-12-01T09:37:48.940 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2022-12-01T09:37:51.045 INFO:teuthology.orchestra.run.smithi085.stdout: Preparing : 1/1 2022-12-01T09:37:51.298 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-12-01T09:37:51.498 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-12-01T09:37:51.711 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-12-01T09:37:51.837 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-12-01T09:37:52.019 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-12-01T09:37:52.197 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-12-01T09:37:52.594 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-12-01T09:37:52.764 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-12-01T09:37:53.161 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-12-01T09:37:53.388 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:37:53.806 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:37:53.919 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2022-12-01T09:37:54.013 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:37:54.133 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-12-01T09:37:54.136 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:37:54.358 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-12-01T09:37:54.371 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:37:54.466 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:37:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-12-01T09:37:54.661 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 10/138 2022-12-01T09:37:54.699 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-12-01T09:37:54.886 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-12-01T09:37:54.915 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-12-01T09:37:55.027 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-12-01T09:37:55.083 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-12-01T09:37:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-12-01T09:37:55.269 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-12-01T09:37:55.467 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-12-01T09:37:55.675 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-12-01T09:37:55.688 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-12-01T09:37:55.821 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:37:55.894 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:37:55.914 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:37:56.137 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 16/138 2022-12-01T09:37:56.210 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 16/138 2022-12-01T09:37:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:37:56.358 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-12-01T09:37:56.502 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:37:56.604 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-12-01T09:37:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:37:56.776 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-12-01T09:37:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:37:56.925 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:37:56.960 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-12-01T09:37:57.079 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-12-01T09:37:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 10/138 2022-12-01T09:37:57.196 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-12-01T09:37:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-12-01T09:37:57.363 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-12-01T09:37:57.514 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-12-01T09:37:57.874 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-12-01T09:37:57.983 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-12-01T09:37:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-12-01T09:37:58.244 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:37:58.287 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-12-01T09:37:58.347 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:37:58.518 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 16/138 2022-12-01T09:37:58.594 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 16/138 2022-12-01T09:37:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-12-01T09:37:58.874 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-12-01T09:37:58.968 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-12-01T09:37:59.037 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-12-01T09:37:59.142 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-12-01T09:37:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-12-01T09:37:59.401 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-12-01T09:37:59.549 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-12-01T09:37:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-12-01T09:37:59.752 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-12-01T09:38:00.067 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-12-01T09:38:00.314 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-12-01T09:38:00.341 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-12-01T09:38:00.493 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x 27/138 2022-12-01T09:38:00.535 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-12-01T09:38:00.622 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8 28/138 2022-12-01T09:38:00.856 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 29/138 2022-12-01T09:38:01.054 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-12-01T09:38:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-12-01T09:38:01.326 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-12-01T09:38:02.056 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-12-01T09:38:02.369 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-12-01T09:38:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-12-01T09:38:02.771 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-12-01T09:38:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x 27/138 2022-12-01T09:38:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8 28/138 2022-12-01T09:38:03.154 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-12-01T09:38:03.169 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 29/138 2022-12-01T09:38:03.385 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-12-01T09:38:04.089 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-12-01T09:38:04.439 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-12-01T09:38:04.695 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-12-01T09:38:04.939 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-12-01T09:38:05.139 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-12-01T09:38:05.234 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-12-01T09:38:05.384 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 37/138 2022-12-01T09:38:05.606 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-12-01T09:38:05.631 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 38/138 2022-12-01T09:38:05.837 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-12-01T09:38:06.028 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-12-01T09:38:06.108 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-12-01T09:38:06.304 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-12-01T09:38:06.382 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-12-01T09:38:06.594 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-12-01T09:38:06.607 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-12-01T09:38:06.744 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-12-01T09:38:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-12-01T09:38:07.002 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-12-01T09:38:07.211 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-12-01T09:38:07.220 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-12-01T09:38:07.391 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-12-01T09:38:07.457 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-12-01T09:38:07.541 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-12-01T09:38:07.655 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-12-01T09:38:07.707 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-12-01T09:38:07.866 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 37/138 2022-12-01T09:38:07.927 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-12-01T09:38:08.101 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 38/138 2022-12-01T09:38:08.142 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-12-01T09:38:08.345 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-12-01T09:38:08.383 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-12-01T09:38:08.517 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-12-01T09:38:08.608 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-12-01T09:38:08.802 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-12-01T09:38:08.820 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-12-01T09:38:08.874 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-12-01T09:38:09.108 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-12-01T09:38:09.120 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-12-01T09:38:09.250 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-12-01T09:38:09.313 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-12-01T09:38:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-12-01T09:38:09.588 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-12-01T09:38:09.709 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-12-01T09:38:09.873 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-12-01T09:38:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-12-01T09:38:10.038 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-12-01T09:38:10.197 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-12-01T09:38:10.256 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-12-01T09:38:10.392 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-12-01T09:38:10.510 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-12-01T09:38:10.617 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-12-01T09:38:10.661 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-12-01T09:38:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-12-01T09:38:10.913 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-12-01T09:38:11.064 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-12-01T09:38:11.226 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-12-01T09:38:11.417 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-12-01T09:38:11.552 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-12-01T09:38:11.606 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-12-01T09:38:11.746 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-12-01T09:38:11.905 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-12-01T09:38:11.975 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-12-01T09:38:12.172 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-12-01T09:38:12.356 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-12-01T09:38:12.411 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-12-01T09:38:12.628 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-12-01T09:38:12.668 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-12-01T09:38:12.826 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-12-01T09:38:12.900 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-12-01T09:38:12.996 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-12-01T09:38:13.043 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-12-01T09:38:13.252 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-12-01T09:38:13.306 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-12-01T09:38:13.396 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-12-01T09:38:13.550 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-12-01T09:38:13.731 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-12-01T09:38:13.737 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-12-01T09:38:13.887 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-12-01T09:38:13.938 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-12-01T09:38:14.109 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-12-01T09:38:14.313 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-12-01T09:38:14.515 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-12-01T09:38:14.520 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-12-01T09:38:14.688 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-12-01T09:38:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-12-01T09:38:14.866 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-12-01T09:38:15.018 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-12-01T09:38:15.080 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-12-01T09:38:15.209 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-12-01T09:38:15.304 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-12-01T09:38:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-12-01T09:38:15.474 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-12-01T09:38:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-12-01T09:38:15.753 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-12-01T09:38:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-12-01T09:38:15.933 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-12-01T09:38:16.058 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-12-01T09:38:16.141 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-12-01T09:38:16.238 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-12-01T09:38:16.331 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-12-01T09:38:16.378 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-12-01T09:38:16.569 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-12-01T09:38:16.755 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-12-01T09:38:16.945 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-12-01T09:38:17.086 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-12-01T09:38:17.230 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-12-01T09:38:17.410 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-12-01T09:38:17.431 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-12-01T09:38:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-12-01T09:38:17.687 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-12-01T09:38:17.830 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-12-01T09:38:17.943 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-12-01T09:38:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-12-01T09:38:18.241 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-12-01T09:38:18.257 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-12-01T09:38:18.424 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-12-01T09:38:18.602 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-12-01T09:38:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-12-01T09:38:18.792 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-12-01T09:38:19.552 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-12-01T09:38:19.701 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 93/138 2022-12-01T09:38:19.743 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-12-01T09:38:19.773 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 93/138 2022-12-01T09:38:20.001 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 94/138 2022-12-01T09:38:20.004 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-12-01T09:38:20.255 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-12-01T09:38:20.533 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-12-01T09:38:20.808 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-12-01T09:38:21.009 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-12-01T09:38:21.792 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-12-01T09:38:22.023 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 93/138 2022-12-01T09:38:22.082 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 93/138 2022-12-01T09:38:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 94/138 2022-12-01T09:38:23.806 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:38:24.026 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:38:24.586 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:38:24.876 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 96/138 2022-12-01T09:38:24.950 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 96/138 2022-12-01T09:38:25.075 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 97/138 2022-12-01T09:38:26.287 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:38:26.473 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:38:27.066 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:38:27.362 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 96/138 2022-12-01T09:38:27.451 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 96/138 2022-12-01T09:38:27.572 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 97/138 2022-12-01T09:38:43.565 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 97/138 2022-12-01T09:38:43.566 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /sys 2022-12-01T09:38:43.566 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /proc 2022-12-01T09:38:43.566 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /dev 2022-12-01T09:38:43.566 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /run 2022-12-01T09:38:43.566 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /mnt 2022-12-01T09:38:43.567 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /var/tmp 2022-12-01T09:38:43.567 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /home 2022-12-01T09:38:43.567 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /tmp 2022-12-01T09:38:43.567 INFO:teuthology.orchestra.run.smithi085.stdout:skipping the directory /dev 2022-12-01T09:38:43.568 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:38:43.973 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 98/138 2022-12-01T09:38:45.470 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 98/138 2022-12-01T09:38:45.681 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 99/138 2022-12-01T09:38:45.900 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 99/138 2022-12-01T09:38:46.081 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 100/138 2022-12-01T09:38:46.465 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 100/138 2022-12-01T09:38:46.673 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 101/138 2022-12-01T09:38:46.690 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 101/138 2022-12-01T09:38:47.265 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 97/138 2022-12-01T09:38:47.265 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2022-12-01T09:38:47.266 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2022-12-01T09:38:47.266 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /dev 2022-12-01T09:38:47.266 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /run 2022-12-01T09:38:47.267 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2022-12-01T09:38:47.267 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2022-12-01T09:38:47.267 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2022-12-01T09:38:47.267 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2022-12-01T09:38:47.268 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /dev 2022-12-01T09:38:47.268 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:38:47.620 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 98/138 2022-12-01T09:38:48.175 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 102/138 2022-12-01T09:38:48.375 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 102/138 2022-12-01T09:38:48.503 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el 103/138 2022-12-01T09:38:48.675 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.e 104/138 2022-12-01T09:38:48.855 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 105/138 2022-12-01T09:38:49.059 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 105/138 2022-12-01T09:38:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 98/138 2022-12-01T09:38:49.364 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 106/138 2022-12-01T09:38:49.533 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 99/138 2022-12-01T09:38:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 99/138 2022-12-01T09:38:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 100/138 2022-12-01T09:38:50.327 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 106/138 2022-12-01T09:38:50.342 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 100/138 2022-12-01T09:38:50.513 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 107/138 2022-12-01T09:38:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 101/138 2022-12-01T09:38:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 101/138 2022-12-01T09:38:50.601 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 107/138 2022-12-01T09:38:50.950 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8 108/138 2022-12-01T09:38:51.134 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 109/138 2022-12-01T09:38:51.168 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 109/138 2022-12-01T09:38:51.363 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 110/138 2022-12-01T09:38:51.374 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 110/138 2022-12-01T09:38:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 102/138 2022-12-01T09:38:52.519 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 102/138 2022-12-01T09:38:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el 103/138 2022-12-01T09:38:52.824 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.e 104/138 2022-12-01T09:38:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 105/138 2022-12-01T09:38:53.072 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 111/138 2022-12-01T09:38:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 105/138 2022-12-01T09:38:53.273 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 112/138 2022-12-01T09:38:53.306 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 112/138 2022-12-01T09:38:53.460 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 106/138 2022-12-01T09:38:53.507 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-676.gf6922c 113/138 2022-12-01T09:38:53.548 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-676.gf6922c 113/138 2022-12-01T09:38:53.696 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 114/138 2022-12-01T09:38:54.047 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 114/138 2022-12-01T09:38:54.205 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 115/138 2022-12-01T09:38:54.485 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 106/138 2022-12-01T09:38:54.643 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 107/138 2022-12-01T09:38:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 107/138 2022-12-01T09:38:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8 108/138 2022-12-01T09:38:55.332 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 109/138 2022-12-01T09:38:55.369 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 109/138 2022-12-01T09:38:55.535 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 110/138 2022-12-01T09:38:55.547 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 110/138 2022-12-01T09:38:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 111/138 2022-12-01T09:38:57.544 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 112/138 2022-12-01T09:38:57.579 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 112/138 2022-12-01T09:38:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-676.gf6922c 113/138 2022-12-01T09:38:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-676.gf6922c 113/138 2022-12-01T09:38:58.010 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 114/138 2022-12-01T09:38:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 114/138 2022-12-01T09:38:58.526 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 115/138 2022-12-01T09:38:59.339 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 115/138 2022-12-01T09:38:59.652 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 116/138 2022-12-01T09:39:00.444 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 117/138 2022-12-01T09:39:00.697 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-12-01T09:39:00.930 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-12-01T09:39:01.070 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-12-01T09:39:01.258 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-12-01T09:39:01.491 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-12-01T09:39:01.740 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_6 122/138 2022-12-01T09:39:01.879 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-12-01T09:39:03.906 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:39:03.966 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 115/138 2022-12-01T09:39:04.155 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:39:04.172 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:39:04.252 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 116/138 2022-12-01T09:39:04.387 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 125/138 2022-12-01T09:39:04.627 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 126/138 2022-12-01T09:39:04.911 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 127/138 2022-12-01T09:39:05.131 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 117/138 2022-12-01T09:39:05.218 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-12-01T09:39:05.412 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-12-01T09:39:05.571 INFO:teuthology.orchestra.run.smithi085.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-12-01T09:39:05.622 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-12-01T09:39:05.775 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-12-01T09:39:05.852 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-12-01T09:39:05.927 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-12-01T09:39:05.980 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-12-01T09:39:06.152 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-12-01T09:39:06.164 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-12-01T09:39:06.230 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-12-01T09:39:06.319 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-12-01T09:39:06.326 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-12-01T09:39:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_6 122/138 2022-12-01T09:39:06.455 INFO:teuthology.orchestra.run.smithi085.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-12-01T09:39:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-12-01T09:39:06.654 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-12-01T09:39:06.654 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-12-01T09:39:06.839 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-12-01T09:39:06.839 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-12-01T09:39:07.105 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-12-01T09:39:07.106 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-12-01T09:39:07.392 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-12-01T09:39:07.568 INFO:teuthology.orchestra.run.smithi085.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:39:08.110 INFO:teuthology.orchestra.run.smithi085.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:39:08.111 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 1/138 2022-12-01T09:39:08.111 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 2/138 2022-12-01T09:39:08.111 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 3/138 2022-12-01T09:39:08.112 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 4/138 2022-12-01T09:39:08.112 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-676.gf6922c 5/138 2022-12-01T09:39:08.112 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 6/138 2022-12-01T09:39:08.112 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:39:08.113 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:39:08.113 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:39:08.113 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 10/138 2022-12-01T09:39:08.113 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 11/138 2022-12-01T09:39:08.114 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 12/138 2022-12-01T09:39:08.114 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 13/138 2022-12-01T09:39:08.114 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_6 14/138 2022-12-01T09:39:08.114 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:39:08.115 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 16/138 2022-12-01T09:39:08.115 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 17/138 2022-12-01T09:39:08.115 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 18/138 2022-12-01T09:39:08.116 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 19/138 2022-12-01T09:39:08.116 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8 20/138 2022-12-01T09:39:08.116 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x 21/138 2022-12-01T09:39:08.116 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 22/138 2022-12-01T09:39:08.117 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 23/138 2022-12-01T09:39:08.117 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 24/138 2022-12-01T09:39:08.117 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 25/138 2022-12-01T09:39:08.117 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 26/138 2022-12-01T09:39:08.118 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 27/138 2022-12-01T09:39:08.118 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 28/138 2022-12-01T09:39:08.118 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.e 29/138 2022-12-01T09:39:08.118 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 30/138 2022-12-01T09:39:08.119 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 31/138 2022-12-01T09:39:08.119 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 32/138 2022-12-01T09:39:08.120 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8 33/138 2022-12-01T09:39:08.120 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 34/138 2022-12-01T09:39:08.120 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el 35/138 2022-12-01T09:39:08.121 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 36/138 2022-12-01T09:39:08.121 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 37/138 2022-12-01T09:39:08.121 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 38/138 2022-12-01T09:39:08.121 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-12-01T09:39:08.122 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-12-01T09:39:08.122 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-12-01T09:39:08.122 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-12-01T09:39:08.122 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-12-01T09:39:08.123 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-12-01T09:39:08.123 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-12-01T09:39:08.123 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-12-01T09:39:08.124 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-12-01T09:39:08.124 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-12-01T09:39:08.124 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-12-01T09:39:08.124 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-12-01T09:39:08.125 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-12-01T09:39:08.125 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-12-01T09:39:08.125 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-12-01T09:39:08.125 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-12-01T09:39:08.125 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-12-01T09:39:08.126 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-12-01T09:39:08.126 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-12-01T09:39:08.126 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-12-01T09:39:08.126 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-12-01T09:39:08.127 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-12-01T09:39:08.127 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-12-01T09:39:08.127 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-12-01T09:39:08.128 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-12-01T09:39:08.128 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-12-01T09:39:08.129 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-12-01T09:39:08.129 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-12-01T09:39:08.129 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-12-01T09:39:08.130 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-12-01T09:39:08.130 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-12-01T09:39:08.131 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-12-01T09:39:08.131 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-12-01T09:39:08.132 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-12-01T09:39:08.132 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-12-01T09:39:08.133 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-12-01T09:39:08.133 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-12-01T09:39:08.134 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-12-01T09:39:08.134 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-12-01T09:39:08.135 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-12-01T09:39:08.135 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-12-01T09:39:08.135 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-12-01T09:39:08.136 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-12-01T09:39:08.136 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-12-01T09:39:08.137 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-12-01T09:39:08.137 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-12-01T09:39:08.138 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-12-01T09:39:08.138 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-12-01T09:39:08.138 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-12-01T09:39:08.139 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-12-01T09:39:08.139 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-12-01T09:39:08.140 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 90/138 2022-12-01T09:39:08.140 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 91/138 2022-12-01T09:39:08.141 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 92/138 2022-12-01T09:39:08.141 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 93/138 2022-12-01T09:39:08.141 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 94/138 2022-12-01T09:39:08.141 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 95/138 2022-12-01T09:39:08.142 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/138 2022-12-01T09:39:08.142 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/138 2022-12-01T09:39:08.142 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-12-01T09:39:08.142 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-12-01T09:39:08.143 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-12-01T09:39:08.143 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-12-01T09:39:08.143 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-12-01T09:39:08.143 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-12-01T09:39:08.144 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-12-01T09:39:08.144 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-12-01T09:39:08.144 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-12-01T09:39:08.144 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-12-01T09:39:08.145 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-12-01T09:39:08.145 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-12-01T09:39:08.145 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-12-01T09:39:08.145 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-12-01T09:39:08.145 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-12-01T09:39:08.146 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-12-01T09:39:08.146 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-12-01T09:39:08.146 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-12-01T09:39:08.146 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-12-01T09:39:08.147 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-12-01T09:39:08.147 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-12-01T09:39:08.147 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-12-01T09:39:08.147 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-12-01T09:39:08.148 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-12-01T09:39:08.148 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-12-01T09:39:08.148 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-12-01T09:39:08.149 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-12-01T09:39:08.149 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-12-01T09:39:08.149 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-12-01T09:39:08.149 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-12-01T09:39:08.150 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-12-01T09:39:08.150 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 129/138 2022-12-01T09:39:08.150 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-12-01T09:39:08.150 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 131/138 2022-12-01T09:39:08.151 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-12-01T09:39:08.151 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-12-01T09:39:08.151 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-12-01T09:39:08.151 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-12-01T09:39:08.152 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-12-01T09:39:08.152 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-12-01T09:39:08.696 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:39:08.970 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:39:08.988 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:39:09.183 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 125/138 2022-12-01T09:39:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 126/138 2022-12-01T09:39:09.688 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 127/138 2022-12-01T09:39:10.030 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-12-01T09:39:10.066 INFO:teuthology.orchestra.run.smithi085.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:39:10.066 INFO:teuthology.orchestra.run.smithi085.stdout:Installed products updated. 2022-12-01T09:39:10.311 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:10.311 INFO:teuthology.orchestra.run.smithi085.stdout:Upgraded: 2022-12-01T09:39:10.312 INFO:teuthology.orchestra.run.smithi085.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-12-01T09:39:10.312 INFO:teuthology.orchestra.run.smithi085.stdout: librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.312 INFO:teuthology.orchestra.run.smithi085.stdout: librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.312 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:39:10.313 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:39:10.313 INFO:teuthology.orchestra.run.smithi085.stdout:Installed: 2022-12-01T09:39:10.313 INFO:teuthology.orchestra.run.smithi085.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-12-01T09:39:10.313 INFO:teuthology.orchestra.run.smithi085.stdout: automake-1.16.1-7.el8.noarch 2022-12-01T09:39:10.314 INFO:teuthology.orchestra.run.smithi085.stdout: bison-3.0.4-10.el8.x86_64 2022-12-01T09:39:10.314 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.314 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.314 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.315 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.315 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.315 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-immutable-object-cache-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.316 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.316 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.317 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.317 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.317 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-diskprediction-local-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.317 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.318 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.318 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.318 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.319 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.319 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.319 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.320 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.320 INFO:teuthology.orchestra.run.smithi085.stdout: ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.320 INFO:teuthology.orchestra.run.smithi085.stdout: cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.320 INFO:teuthology.orchestra.run.smithi085.stdout: cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.321 INFO:teuthology.orchestra.run.smithi085.stdout: cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:10.321 INFO:teuthology.orchestra.run.smithi085.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-12-01T09:39:10.321 INFO:teuthology.orchestra.run.smithi085.stdout: flex-2.6.1-9.el8.x86_64 2022-12-01T09:39:10.322 INFO:teuthology.orchestra.run.smithi085.stdout: fuse-2.9.7-15.el8.x86_64 2022-12-01T09:39:10.322 INFO:teuthology.orchestra.run.smithi085.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-12-01T09:39:10.322 INFO:teuthology.orchestra.run.smithi085.stdout: jq-1.6-3.el8.x86_64 2022-12-01T09:39:10.323 INFO:teuthology.orchestra.run.smithi085.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-12-01T09:39:10.323 INFO:teuthology.orchestra.run.smithi085.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-12-01T09:39:10.323 INFO:teuthology.orchestra.run.smithi085.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-12-01T09:39:10.324 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.324 INFO:teuthology.orchestra.run.smithi085.stdout: libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.324 INFO:teuthology.orchestra.run.smithi085.stdout: libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.324 INFO:teuthology.orchestra.run.smithi085.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-12-01T09:39:10.324 INFO:teuthology.orchestra.run.smithi085.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-12-01T09:39:10.325 INFO:teuthology.orchestra.run.smithi085.stdout: liboath-2.6.2-3.el8.x86_64 2022-12-01T09:39:10.325 INFO:teuthology.orchestra.run.smithi085.stdout: libpq-13.5-1.el8.x86_64 2022-12-01T09:39:10.325 INFO:teuthology.orchestra.run.smithi085.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-12-01T09:39:10.325 INFO:teuthology.orchestra.run.smithi085.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-12-01T09:39:10.326 INFO:teuthology.orchestra.run.smithi085.stdout: librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.326 INFO:teuthology.orchestra.run.smithi085.stdout: libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.326 INFO:teuthology.orchestra.run.smithi085.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-12-01T09:39:10.326 INFO:teuthology.orchestra.run.smithi085.stdout: librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.327 INFO:teuthology.orchestra.run.smithi085.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-12-01T09:39:10.327 INFO:teuthology.orchestra.run.smithi085.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-12-01T09:39:10.327 INFO:teuthology.orchestra.run.smithi085.stdout: libtool-2.4.6-25.el8.x86_64 2022-12-01T09:39:10.327 INFO:teuthology.orchestra.run.smithi085.stdout: libunwind-1.3.1-3.el8.x86_64 2022-12-01T09:39:10.327 INFO:teuthology.orchestra.run.smithi085.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-12-01T09:39:10.328 INFO:teuthology.orchestra.run.smithi085.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-12-01T09:39:10.328 INFO:teuthology.orchestra.run.smithi085.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-12-01T09:39:10.328 INFO:teuthology.orchestra.run.smithi085.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-12-01T09:39:10.328 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-0.3.15-3.el8.x86_64 2022-12-01T09:39:10.329 INFO:teuthology.orchestra.run.smithi085.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-12-01T09:39:10.329 INFO:teuthology.orchestra.run.smithi085.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:39:10.329 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-12-01T09:39:10.329 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-12-01T09:39:10.330 INFO:teuthology.orchestra.run.smithi085.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-12-01T09:39:10.330 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:39:10.330 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:39:10.330 INFO:teuthology.orchestra.run.smithi085.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:39:10.331 INFO:teuthology.orchestra.run.smithi085.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-12-01T09:39:10.331 INFO:teuthology.orchestra.run.smithi085.stdout: python3-babel-2.5.1-7.el8.noarch 2022-12-01T09:39:10.331 INFO:teuthology.orchestra.run.smithi085.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-12-01T09:39:10.331 INFO:teuthology.orchestra.run.smithi085.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-12-01T09:39:10.331 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-12-01T09:39:10.332 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.332 INFO:teuthology.orchestra.run.smithi085.stdout: python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.332 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.332 INFO:teuthology.orchestra.run.smithi085.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-12-01T09:39:10.333 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-12-01T09:39:10.333 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-12-01T09:39:10.333 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-12-01T09:39:10.333 INFO:teuthology.orchestra.run.smithi085.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-12-01T09:39:10.333 INFO:teuthology.orchestra.run.smithi085.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-12-01T09:39:10.334 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-6.2-6.el8.noarch 2022-12-01T09:39:10.334 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-12-01T09:39:10.334 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-12-01T09:39:10.334 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-12-01T09:39:10.335 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-12-01T09:39:10.335 INFO:teuthology.orchestra.run.smithi085.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-12-01T09:39:10.335 INFO:teuthology.orchestra.run.smithi085.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-12-01T09:39:10.335 INFO:teuthology.orchestra.run.smithi085.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-12-01T09:39:10.335 INFO:teuthology.orchestra.run.smithi085.stdout: python3-mako-1.0.6-13.el8.noarch 2022-12-01T09:39:10.336 INFO:teuthology.orchestra.run.smithi085.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-12-01T09:39:10.336 INFO:teuthology.orchestra.run.smithi085.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-12-01T09:39:10.336 INFO:teuthology.orchestra.run.smithi085.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-12-01T09:39:10.336 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-12-01T09:39:10.337 INFO:teuthology.orchestra.run.smithi085.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-12-01T09:39:10.337 INFO:teuthology.orchestra.run.smithi085.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-12-01T09:39:10.337 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-12-01T09:39:10.337 INFO:teuthology.orchestra.run.smithi085.stdout: python3-portend-2.6-1.el8.noarch 2022-12-01T09:39:10.338 INFO:teuthology.orchestra.run.smithi085.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-12-01T09:39:10.338 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-12-01T09:39:10.338 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-12-01T09:39:10.338 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-12-01T09:39:10.338 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pycparser-2.14-14.el8.noarch 2022-12-01T09:39:10.339 INFO:teuthology.orchestra.run.smithi085.stdout: python3-pytz-2017.2-9.el8.noarch 2022-12-01T09:39:10.339 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.339 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.339 INFO:teuthology.orchestra.run.smithi085.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-12-01T09:39:10.340 INFO:teuthology.orchestra.run.smithi085.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-12-01T09:39:10.340 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.340 INFO:teuthology.orchestra.run.smithi085.stdout: python3-routes-2.4.1-12.el8.noarch 2022-12-01T09:39:10.340 INFO:teuthology.orchestra.run.smithi085.stdout: python3-rsa-4.8-1.el8.noarch 2022-12-01T09:39:10.340 INFO:teuthology.orchestra.run.smithi085.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-12-01T09:39:10.341 INFO:teuthology.orchestra.run.smithi085.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-12-01T09:39:10.341 INFO:teuthology.orchestra.run.smithi085.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-12-01T09:39:10.341 INFO:teuthology.orchestra.run.smithi085.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-12-01T09:39:10.341 INFO:teuthology.orchestra.run.smithi085.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-12-01T09:39:10.341 INFO:teuthology.orchestra.run.smithi085.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-12-01T09:39:10.342 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-12-01T09:39:10.342 INFO:teuthology.orchestra.run.smithi085.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-12-01T09:39:10.342 INFO:teuthology.orchestra.run.smithi085.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-12-01T09:39:10.342 INFO:teuthology.orchestra.run.smithi085.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-12-01T09:39:10.343 INFO:teuthology.orchestra.run.smithi085.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-12-01T09:39:10.343 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.343 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.343 INFO:teuthology.orchestra.run.smithi085.stdout: rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:10.343 INFO:teuthology.orchestra.run.smithi085.stdout: socat-1.7.4.1-1.el8.x86_64 2022-12-01T09:39:10.344 INFO:teuthology.orchestra.run.smithi085.stdout: thrift-0.13.0-2.el8.x86_64 2022-12-01T09:39:10.344 INFO:teuthology.orchestra.run.smithi085.stdout: uuid-1.6.2-43.el8.x86_64 2022-12-01T09:39:10.344 INFO:teuthology.orchestra.run.smithi085.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-12-01T09:39:10.344 INFO:teuthology.orchestra.run.smithi085.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-12-01T09:39:10.345 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-12-01T09:39:10.346 INFO:teuthology.orchestra.run.smithi085.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-12-01T09:39:10.346 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:10.346 INFO:teuthology.orchestra.run.smithi085.stdout:Complete! 2022-12-01T09:39:10.522 DEBUG:teuthology.parallel:result is None 2022-12-01T09:39:10.536 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-12-01T09:39:10.612 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-12-01T09:39:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-12-01T09:39:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-12-01T09:39:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-12-01T09:39:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-12-01T09:39:11.179 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-12-01T09:39:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-12-01T09:39:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-12-01T09:39:11.544 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-12-01T09:39:11.545 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-12-01T09:39:11.786 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-12-01T09:39:11.786 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-12-01T09:39:12.072 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-12-01T09:39:12.249 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:39:12.914 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:39:12.914 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 1/138 2022-12-01T09:39:12.914 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 2/138 2022-12-01T09:39:12.914 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 3/138 2022-12-01T09:39:12.915 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 4/138 2022-12-01T09:39:12.915 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-676.gf6922c 5/138 2022-12-01T09:39:12.915 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 6/138 2022-12-01T09:39:12.915 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:39:12.916 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:39:12.916 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:39:12.916 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 10/138 2022-12-01T09:39:12.916 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 11/138 2022-12-01T09:39:12.917 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 12/138 2022-12-01T09:39:12.917 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 13/138 2022-12-01T09:39:12.917 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_6 14/138 2022-12-01T09:39:12.917 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:39:12.917 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 16/138 2022-12-01T09:39:12.918 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 17/138 2022-12-01T09:39:12.918 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 18/138 2022-12-01T09:39:12.918 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 19/138 2022-12-01T09:39:12.918 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8 20/138 2022-12-01T09:39:12.918 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x 21/138 2022-12-01T09:39:12.919 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 22/138 2022-12-01T09:39:12.919 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 23/138 2022-12-01T09:39:12.919 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 24/138 2022-12-01T09:39:12.919 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 25/138 2022-12-01T09:39:12.920 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 26/138 2022-12-01T09:39:12.920 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 27/138 2022-12-01T09:39:12.920 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 28/138 2022-12-01T09:39:12.920 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.e 29/138 2022-12-01T09:39:12.920 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 30/138 2022-12-01T09:39:12.921 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 31/138 2022-12-01T09:39:12.921 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 32/138 2022-12-01T09:39:12.921 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8 33/138 2022-12-01T09:39:12.921 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 34/138 2022-12-01T09:39:12.921 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el 35/138 2022-12-01T09:39:12.922 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 36/138 2022-12-01T09:39:12.922 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 37/138 2022-12-01T09:39:12.922 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 38/138 2022-12-01T09:39:12.922 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-12-01T09:39:12.923 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-12-01T09:39:12.923 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-12-01T09:39:12.923 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-12-01T09:39:12.923 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-12-01T09:39:12.923 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-12-01T09:39:12.924 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-12-01T09:39:12.924 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-12-01T09:39:12.924 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-12-01T09:39:12.924 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-12-01T09:39:12.924 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-12-01T09:39:12.925 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-12-01T09:39:12.925 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-12-01T09:39:12.926 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-12-01T09:39:12.926 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-12-01T09:39:12.926 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-12-01T09:39:12.926 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-12-01T09:39:12.927 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-12-01T09:39:12.927 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-12-01T09:39:12.927 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-12-01T09:39:12.927 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-12-01T09:39:12.928 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-12-01T09:39:12.928 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-12-01T09:39:12.928 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-12-01T09:39:12.928 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-12-01T09:39:12.928 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-12-01T09:39:12.929 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-12-01T09:39:12.929 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-12-01T09:39:12.929 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-12-01T09:39:12.929 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-12-01T09:39:12.930 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-12-01T09:39:12.930 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-12-01T09:39:12.930 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-12-01T09:39:12.930 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-12-01T09:39:12.930 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-12-01T09:39:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-12-01T09:39:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-12-01T09:39:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-12-01T09:39:12.931 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-12-01T09:39:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-12-01T09:39:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-12-01T09:39:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-12-01T09:39:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-12-01T09:39:12.932 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-12-01T09:39:12.933 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-12-01T09:39:12.933 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-12-01T09:39:12.933 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-12-01T09:39:12.933 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-12-01T09:39:12.934 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-12-01T09:39:12.934 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-12-01T09:39:12.934 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-12-01T09:39:12.934 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 90/138 2022-12-01T09:39:12.934 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 91/138 2022-12-01T09:39:12.935 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 92/138 2022-12-01T09:39:12.935 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 93/138 2022-12-01T09:39:12.935 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 94/138 2022-12-01T09:39:12.935 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 95/138 2022-12-01T09:39:12.935 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/138 2022-12-01T09:39:12.936 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/138 2022-12-01T09:39:12.936 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-12-01T09:39:12.936 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-12-01T09:39:12.936 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-12-01T09:39:12.937 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-12-01T09:39:12.937 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-12-01T09:39:12.937 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-12-01T09:39:12.937 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-12-01T09:39:12.937 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-12-01T09:39:12.938 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-12-01T09:39:12.938 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-12-01T09:39:12.938 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-12-01T09:39:12.938 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-12-01T09:39:12.938 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-12-01T09:39:12.939 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-12-01T09:39:12.939 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-12-01T09:39:12.939 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-12-01T09:39:12.939 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-12-01T09:39:12.940 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-12-01T09:39:12.940 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-12-01T09:39:12.940 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-12-01T09:39:12.940 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-12-01T09:39:12.940 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-12-01T09:39:12.941 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-12-01T09:39:12.941 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-12-01T09:39:12.941 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-12-01T09:39:12.941 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-12-01T09:39:12.941 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-12-01T09:39:12.942 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-12-01T09:39:12.942 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-12-01T09:39:12.942 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-12-01T09:39:12.942 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-12-01T09:39:12.943 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 129/138 2022-12-01T09:39:12.943 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-12-01T09:39:12.943 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 131/138 2022-12-01T09:39:12.943 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-12-01T09:39:12.943 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-12-01T09:39:12.944 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-12-01T09:39:12.944 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-12-01T09:39:12.944 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-12-01T09:39:12.944 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-12-01T09:39:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:39:14.806 INFO:teuthology.orchestra.run.smithi002.stdout:Installed products updated. 2022-12-01T09:39:15.077 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:15.077 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2022-12-01T09:39:15.077 INFO:teuthology.orchestra.run.smithi002.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-12-01T09:39:15.078 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.078 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.079 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:39:15.080 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:39:15.080 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2022-12-01T09:39:15.081 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-12-01T09:39:15.081 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.1-7.el8.noarch 2022-12-01T09:39:15.081 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.0.4-10.el8.x86_64 2022-12-01T09:39:15.081 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.082 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.082 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.082 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.082 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.083 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.083 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.084 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.084 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.084 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.084 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.085 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.085 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.085 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.085 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.085 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.086 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.086 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.086 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.086 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.087 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.087 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.087 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:39:15.087 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-12-01T09:39:15.087 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.1-9.el8.x86_64 2022-12-01T09:39:15.088 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.7-15.el8.x86_64 2022-12-01T09:39:15.088 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-12-01T09:39:15.088 INFO:teuthology.orchestra.run.smithi002.stdout: jq-1.6-3.el8.x86_64 2022-12-01T09:39:15.088 INFO:teuthology.orchestra.run.smithi002.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-12-01T09:39:15.089 INFO:teuthology.orchestra.run.smithi002.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-12-01T09:39:15.089 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-12-01T09:39:15.089 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.089 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.090 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.090 INFO:teuthology.orchestra.run.smithi002.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-12-01T09:39:15.090 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-12-01T09:39:15.091 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.2-3.el8.x86_64 2022-12-01T09:39:15.091 INFO:teuthology.orchestra.run.smithi002.stdout: libpq-13.5-1.el8.x86_64 2022-12-01T09:39:15.092 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-12-01T09:39:15.092 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-12-01T09:39:15.093 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.093 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.093 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-12-01T09:39:15.094 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.094 INFO:teuthology.orchestra.run.smithi002.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-12-01T09:39:15.094 INFO:teuthology.orchestra.run.smithi002.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-12-01T09:39:15.094 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-25.el8.x86_64 2022-12-01T09:39:15.094 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.3.1-3.el8.x86_64 2022-12-01T09:39:15.095 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-12-01T09:39:15.095 INFO:teuthology.orchestra.run.smithi002.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-12-01T09:39:15.095 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-12-01T09:39:15.095 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-12-01T09:39:15.096 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.15-3.el8.x86_64 2022-12-01T09:39:15.096 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-12-01T09:39:15.096 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:39:15.097 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-12-01T09:39:15.097 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-12-01T09:39:15.097 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-12-01T09:39:15.098 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:39:15.098 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:39:15.099 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:39:15.099 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-12-01T09:39:15.099 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.5.1-7.el8.noarch 2022-12-01T09:39:15.100 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-12-01T09:39:15.100 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-12-01T09:39:15.101 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-12-01T09:39:15.101 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.101 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.102 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.102 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-12-01T09:39:15.103 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-12-01T09:39:15.103 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-12-01T09:39:15.104 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-12-01T09:39:15.104 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-12-01T09:39:15.104 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-12-01T09:39:15.105 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-6.2-6.el8.noarch 2022-12-01T09:39:15.105 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-12-01T09:39:15.106 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-12-01T09:39:15.106 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-12-01T09:39:15.107 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-12-01T09:39:15.107 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-12-01T09:39:15.108 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-12-01T09:39:15.108 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-12-01T09:39:15.109 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.0.6-13.el8.noarch 2022-12-01T09:39:15.109 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-12-01T09:39:15.109 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-12-01T09:39:15.110 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-12-01T09:39:15.110 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-12-01T09:39:15.111 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-12-01T09:39:15.111 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-12-01T09:39:15.112 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-12-01T09:39:15.112 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-2.6-1.el8.noarch 2022-12-01T09:39:15.113 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-12-01T09:39:15.113 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-12-01T09:39:15.113 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-12-01T09:39:15.114 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-12-01T09:39:15.114 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.14-14.el8.noarch 2022-12-01T09:39:15.115 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2017.2-9.el8.noarch 2022-12-01T09:39:15.115 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.116 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.116 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-12-01T09:39:15.116 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-12-01T09:39:15.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.4.1-12.el8.noarch 2022-12-01T09:39:15.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.8-1.el8.noarch 2022-12-01T09:39:15.117 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-12-01T09:39:15.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-12-01T09:39:15.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-12-01T09:39:15.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-12-01T09:39:15.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-12-01T09:39:15.118 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-12-01T09:39:15.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-12-01T09:39:15.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-12-01T09:39:15.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-12-01T09:39:15.119 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-12-01T09:39:15.120 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-12-01T09:39:15.120 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.120 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.120 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:39:15.121 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-1.el8.x86_64 2022-12-01T09:39:15.121 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.13.0-2.el8.x86_64 2022-12-01T09:39:15.121 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-43.el8.x86_64 2022-12-01T09:39:15.121 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-12-01T09:39:15.122 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-12-01T09:39:15.123 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-12-01T09:39:15.123 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:15.123 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2022-12-01T09:39:15.258 DEBUG:teuthology.parallel:result is None 2022-12-01T09:39:15.259 DEBUG:teuthology.orchestra.run.smithi002:> sudo lsb_release -is 2022-12-01T09:39:15.320 INFO:teuthology.orchestra.run.smithi002.stdout:RedHatEnterprise 2022-12-01T09:39:15.321 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:39:15.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:39:15.636 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-12-01T09:39:15.660 INFO:teuthology.orchestra.run.smithi002.stdout:18.0.0-676.gf6922ca2.el8 2022-12-01T09:39:15.661 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-676.gf6922ca2.el8 2022-12-01T09:39:15.662 INFO:teuthology.task.install:The correct ceph version 18.0.0-676.gf6922ca2 is installed. 2022-12-01T09:39:15.663 DEBUG:teuthology.orchestra.run.smithi085:> sudo lsb_release -is 2022-12-01T09:39:15.728 INFO:teuthology.orchestra.run.smithi085.stdout:RedHatEnterprise 2022-12-01T09:39:15.730 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:39:15.730 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:39:16.057 DEBUG:teuthology.orchestra.run.smithi085:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-12-01T09:39:16.085 INFO:teuthology.orchestra.run.smithi085.stdout:18.0.0-676.gf6922ca2.el8 2022-12-01T09:39:16.085 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-676.gf6922ca2.el8 2022-12-01T09:39:16.086 INFO:teuthology.task.install:The correct ceph version 18.0.0-676.gf6922ca2 is installed. 2022-12-01T09:39:16.086 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-12-01T09:39:16.087 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:16.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-12-01T09:39:16.115 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:16.115 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-12-01T09:39:16.157 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-12-01T09:39:16.158 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:16.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2022-12-01T09:39:16.187 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-12-01T09:39:16.254 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:16.255 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/daemon-helper 2022-12-01T09:39:16.288 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-12-01T09:39:16.359 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-12-01T09:39:16.360 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:16.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2022-12-01T09:39:16.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-12-01T09:39:16.456 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:16.457 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/usr/bin/adjust-ulimits 2022-12-01T09:39:16.489 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-12-01T09:39:16.558 INFO:teuthology.run_tasks:Running task ceph... 2022-12-01T09:39:16.640 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-12-01T09:39:16.641 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /var/log/ceph 2022-12-01T09:39:16.644 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod 777 /var/log/ceph 2022-12-01T09:39:16.673 INFO:tasks.ceph:Disabling ceph logrotate... 2022-12-01T09:39:16.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/logrotate.d/ceph 2022-12-01T09:39:16.712 DEBUG:teuthology.orchestra.run.smithi085:> sudo rm -f -- /etc/logrotate.d/ceph 2022-12-01T09:39:16.741 INFO:tasks.ceph:Creating extra log directories... 2022-12-01T09:39:16.742 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-12-01T09:39:16.781 DEBUG:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-12-01T09:39:16.819 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-12-01T09:39:16.819 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-12-01T09:39:16.820 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-12-01T09:39:16.821 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:16.821 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-12-01T09:39:16.865 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-12-01T09:39:16.981 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-12-01T09:39:17.010 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:17.011 DEBUG:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-12-01T09:39:17.029 DEBUG:teuthology.orchestra.run.smithi085:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-12-01T09:39:17.146 DEBUG:teuthology.orchestra.run.smithi085:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-12-01T09:39:17.182 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-12-01T09:39:17.182 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-12-01T09:39:17.183 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090974', 'branch': 'wip-khiremat-47893-test-mgr-finisher-thread-5', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7090974', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-khiremat-47893-test-mgr-finisher-thread-5'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-khiremat-47893-test-mgr-finisher-thread-5', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}}, 'owner': 'scheduled_khiremat@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'wip-khiremat-47893-test-mgr-finisher-thread-5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_f6922ca21f63dad2b198f92826f75d268f12fc97/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi002.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCC4yytdRLVQkZ4snbnZWiEij3REBJ1sE4lgmo1QdW3ofIEzQouVkQVZzZ6DL0KEOyr1LRpMldVazXwvPuuSQ1s=', 'smithi085.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ7ZZv9k+3flOBXaHgwJy0MWsDqBrqkE+tGtcRaq7K3AlfFqCqg6/irFVeiXAnHIERXBnwt2HfwDt8tsjgemxGN6pCphEas8ApcMObbcNFIxXKb2VOPfAd7/7K8FrcQD8fDUodOt4YTDpL+3hc2+BR2b08neNjBCKn+gL3/3dYEAt/3IJSVFbwZyU2VKlqlCFwA0cszmgXAxx4r9ZnMEfxtttiiwKOm5jNNJvwQvbJoxt2ecL12DAYtUBaYunggGraZvSNJgiN/vEy6XHNdhLPuBwgUTpw0tL5LZCHilphTWSDqYXJmOgBohAIJitB1poDkSRQF+ORVlpFy/v08mbScYHcGyyigMkERheSfpAs7AmFR8dQpZ2RP820OlRw78Zy0RaX7Dz0VzFiF/WuE690NtN/SSJz+srXhJSCO9vTNLM4RQ12M0/HcNha/L2F7nJyie9om70QDajcwtoLBziHOc3gWm71ge901Zud7RuyOOH3Jd56m3oD+d7OgviOYEM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-24_05:02:42', 'tube': 'smithi', 'user': 'khiremat', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23126'} 2022-12-01T09:39:17.185 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-12-01T09:39:17.188 DEBUG:teuthology.orchestra.run.smithi085:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-12-01T09:39:17.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/run/ceph 2022-12-01T09:39:17.246 DEBUG:teuthology.orchestra.run.smithi085:> sudo install -d -m0777 -- /var/run/ceph 2022-12-01T09:39:17.298 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:17.298 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2022-12-01T09:39:17.330 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-12-01T09:39:17.330 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2022-12-01T09:39:17.388 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-12-01T09:39:17.389 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:39:17.389 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 367438 Links: 1 2022-12-01T09:39:17.389 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:39:17.390 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:39:17.390 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2022-12-01 09:39:12.531664778 +0000 2022-12-01T09:39:17.391 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2022-12-01 09:36:11.062037465 +0000 2022-12-01T09:39:17.391 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2022-12-01 09:36:11.062037465 +0000 2022-12-01T09:39:17.391 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2022-12-01T09:39:17.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-12-01T09:39:17.455 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2022-12-01T09:39:17.456 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2022-12-01T09:39:17.456 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000101474 s, 5.0 MB/s 2022-12-01T09:39:17.458 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-12-01T09:39:17.517 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2022-12-01T09:39:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-12-01T09:39:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:39:17.575 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 367525 Links: 1 2022-12-01T09:39:17.576 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:39:17.576 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:39:17.576 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2022-12-01 09:39:12.531664778 +0000 2022-12-01T09:39:17.577 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2022-12-01 09:36:11.608024500 +0000 2022-12-01T09:39:17.577 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2022-12-01 09:36:11.608024500 +0000 2022-12-01T09:39:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2022-12-01T09:39:17.577 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-12-01T09:39:17.641 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2022-12-01T09:39:17.641 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2022-12-01T09:39:17.641 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000103377 s, 5.0 MB/s 2022-12-01T09:39:17.642 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-12-01T09:39:17.701 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2022-12-01T09:39:17.758 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-12-01T09:39:17.759 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:39:17.759 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 362249 Links: 1 2022-12-01T09:39:17.759 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:39:17.760 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:39:17.760 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2022-12-01 09:39:12.531664778 +0000 2022-12-01T09:39:17.760 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2022-12-01 09:36:12.186010775 +0000 2022-12-01T09:39:17.760 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2022-12-01 09:36:12.186010775 +0000 2022-12-01T09:39:17.760 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2022-12-01T09:39:17.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-12-01T09:39:17.824 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2022-12-01T09:39:17.825 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2022-12-01T09:39:17.825 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000101804 s, 5.0 MB/s 2022-12-01T09:39:17.826 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-12-01T09:39:17.885 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2022-12-01T09:39:17.942 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-12-01T09:39:17.943 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:39:17.943 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 367804 Links: 1 2022-12-01T09:39:17.943 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:39:17.944 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:39:17.944 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2022-12-01 09:39:12.532664754 +0000 2022-12-01T09:39:17.944 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2022-12-01 09:36:12.734997739 +0000 2022-12-01T09:39:17.944 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2022-12-01 09:36:12.734997739 +0000 2022-12-01T09:39:17.944 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2022-12-01T09:39:17.945 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-12-01T09:39:18.011 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2022-12-01T09:39:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2022-12-01T09:39:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 9.0991e-05 s, 5.6 MB/s 2022-12-01T09:39:18.037 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-12-01T09:39:18.096 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:18.096 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:18.097 DEBUG:teuthology.orchestra.run.smithi085:> dd if=/scratch_devs of=/dev/stdout 2022-12-01T09:39:18.120 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-12-01T09:39:18.121 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_1 2022-12-01T09:39:18.180 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-12-01T09:39:18.180 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:39:18.181 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 359734 Links: 1 2022-12-01T09:39:18.181 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:39:18.181 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:39:18.181 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2022-12-01 09:39:07.796587734 +0000 2022-12-01T09:39:18.182 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2022-12-01 09:35:56.487060277 +0000 2022-12-01T09:39:18.182 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2022-12-01 09:35:56.487060277 +0000 2022-12-01T09:39:18.182 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2022-12-01T09:39:18.182 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-12-01T09:39:18.247 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2022-12-01T09:39:18.248 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2022-12-01T09:39:18.248 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000229356 s, 2.2 MB/s 2022-12-01T09:39:18.249 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-12-01T09:39:18.310 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_2 2022-12-01T09:39:18.368 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-12-01T09:39:18.369 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:39:18.369 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 367852 Links: 1 2022-12-01T09:39:18.369 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:39:18.370 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:39:18.370 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2022-12-01 09:39:07.796587734 +0000 2022-12-01T09:39:18.370 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2022-12-01 09:35:57.051049644 +0000 2022-12-01T09:39:18.370 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2022-12-01 09:35:57.051049644 +0000 2022-12-01T09:39:18.370 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2022-12-01T09:39:18.371 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-12-01T09:39:18.435 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2022-12-01T09:39:18.435 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2022-12-01T09:39:18.436 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000211797 s, 2.4 MB/s 2022-12-01T09:39:18.437 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-12-01T09:39:18.496 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_3 2022-12-01T09:39:18.553 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-12-01T09:39:18.554 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:39:18.554 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 356135 Links: 1 2022-12-01T09:39:18.554 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:39:18.554 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:39:18.555 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2022-12-01 09:39:07.796587734 +0000 2022-12-01T09:39:18.555 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2022-12-01 09:35:57.580039671 +0000 2022-12-01T09:39:18.555 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2022-12-01 09:35:57.580039671 +0000 2022-12-01T09:39:18.555 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2022-12-01T09:39:18.556 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-12-01T09:39:18.622 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2022-12-01T09:39:18.623 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2022-12-01T09:39:18.623 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000192538 s, 2.7 MB/s 2022-12-01T09:39:18.624 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-12-01T09:39:18.683 DEBUG:teuthology.orchestra.run.smithi085:> stat /dev/vg_nvme/lv_4 2022-12-01T09:39:18.740 INFO:teuthology.orchestra.run.smithi085.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-12-01T09:39:18.740 INFO:teuthology.orchestra.run.smithi085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:39:18.740 INFO:teuthology.orchestra.run.smithi085.stdout:Device: 6h/6d Inode: 363510 Links: 1 2022-12-01T09:39:18.741 INFO:teuthology.orchestra.run.smithi085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:39:18.741 INFO:teuthology.orchestra.run.smithi085.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:39:18.741 INFO:teuthology.orchestra.run.smithi085.stdout:Access: 2022-12-01 09:39:07.796587734 +0000 2022-12-01T09:39:18.741 INFO:teuthology.orchestra.run.smithi085.stdout:Modify: 2022-12-01 09:35:58.117029547 +0000 2022-12-01T09:39:18.742 INFO:teuthology.orchestra.run.smithi085.stdout:Change: 2022-12-01 09:35:58.117029547 +0000 2022-12-01T09:39:18.742 INFO:teuthology.orchestra.run.smithi085.stdout: Birth: - 2022-12-01T09:39:18.743 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-12-01T09:39:18.804 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records in 2022-12-01T09:39:18.805 INFO:teuthology.orchestra.run.smithi085.stderr:1+0 records out 2022-12-01T09:39:18.805 INFO:teuthology.orchestra.run.smithi085.stderr:512 bytes copied, 0.000215895 s, 2.4 MB/s 2022-12-01T09:39:18.806 DEBUG:teuthology.orchestra.run.smithi085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-12-01T09:39:18.863 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:18.864 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi085.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-12-01T09:39:18.864 INFO:tasks.ceph:Generating config... 2022-12-01T09:39:18.866 INFO:tasks.ceph:[client] client mount timeout = 600 2022-12-01T09:39:18.867 INFO:tasks.ceph:[client] debug client = 20 2022-12-01T09:39:18.867 INFO:tasks.ceph:[client] debug ms = 1 2022-12-01T09:39:18.868 INFO:tasks.ceph:[client] pid file = 2022-12-01T09:39:18.869 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-12-01T09:39:18.869 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-12-01T09:39:18.870 INFO:tasks.ceph:[global] ms die on skipped message = False 2022-12-01T09:39:18.870 INFO:tasks.ceph:[mds] debug mds = 20 2022-12-01T09:39:18.871 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-12-01T09:39:18.872 INFO:tasks.ceph:[mds] debug ms = 1 2022-12-01T09:39:18.872 INFO:tasks.ceph:[mds] mds debug frag = True 2022-12-01T09:39:18.873 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-12-01T09:39:18.874 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-12-01T09:39:18.874 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-12-01T09:39:18.875 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-12-01T09:39:18.875 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-12-01T09:39:18.876 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-12-01T09:39:18.876 INFO:tasks.ceph:[mgr] debug client = 20 2022-12-01T09:39:18.877 INFO:tasks.ceph:[mgr] debug finisher = 20 2022-12-01T09:39:18.877 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-12-01T09:39:18.878 INFO:tasks.ceph:[mgr] debug ms = 1 2022-12-01T09:39:18.879 INFO:tasks.ceph:[mon] debug mon = 20 2022-12-01T09:39:18.879 INFO:tasks.ceph:[mon] debug ms = 1 2022-12-01T09:39:18.880 INFO:tasks.ceph:[mon] debug paxos = 20 2022-12-01T09:39:18.881 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-12-01T09:39:18.881 INFO:tasks.ceph:[osd] bdev async discard = True 2022-12-01T09:39:18.882 INFO:tasks.ceph:[osd] bdev enable discard = True 2022-12-01T09:39:18.882 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2022-12-01T09:39:18.883 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-12-01T09:39:18.884 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-12-01T09:39:18.884 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2022-12-01T09:39:18.885 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2022-12-01T09:39:18.885 INFO:tasks.ceph:[osd] debug ms = 1 2022-12-01T09:39:18.886 INFO:tasks.ceph:[osd] debug osd = 20 2022-12-01T09:39:18.886 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2022-12-01T09:39:18.887 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-12-01T09:39:18.887 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-12-01T09:39:18.888 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-12-01T09:39:18.889 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-12-01T09:39:18.889 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-12-01T09:39:18.890 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-12-01T09:39:18.891 INFO:tasks.ceph:Setting up mon.a... 2022-12-01T09:39:18.891 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-12-01T09:39:18.947 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.keyring 2022-12-01T09:39:18.950 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-12-01T09:39:19.001 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-12-01T09:39:19.032 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.2'), ('mon.b', '172.21.15.85'), ('mon.c', '[v2:172.21.15.85:3301,v1:172.21.15.85:6790]')] 2022-12-01T09:39:19.033 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.2,172.21.15.85,[v2:172.21.15.85:3301,v1:172.21.15.85:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-12-01T09:39:19.034 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:19.035 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-12-01T09:39:19.091 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.2 --add b 172.21.15.85 --addv c '[v2:172.21.15.85:3301,v1:172.21.15.85:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:19.182 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:19.182 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: generated fsid 3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803 2022-12-01T09:39:19.183 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = pacific 2022-12-01T09:39:19.183 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 0 2022-12-01T09:39:19.183 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803 2022-12-01T09:39:19.183 INFO:teuthology.orchestra.run.smithi002.stdout:last_changed 2022-12-01T09:39:19.180269+0000 2022-12-01T09:39:19.184 INFO:teuthology.orchestra.run.smithi002.stdout:created 2022-12-01T09:39:19.180269+0000 2022-12-01T09:39:19.184 INFO:teuthology.orchestra.run.smithi002.stdout:min_mon_release 16 (pacific) 2022-12-01T09:39:19.184 INFO:teuthology.orchestra.run.smithi002.stdout:election_strategy: 1 2022-12-01T09:39:19.184 INFO:teuthology.orchestra.run.smithi002.stdout:0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2022-12-01T09:39:19.185 INFO:teuthology.orchestra.run.smithi002.stdout:1: [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] mon.b 2022-12-01T09:39:19.185 INFO:teuthology.orchestra.run.smithi002.stdout:2: [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] mon.c 2022-12-01T09:39:19.185 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-12-01T09:39:19.186 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-12-01T09:39:19.204 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803... 2022-12-01T09:39:19.206 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-12-01T09:39:19.247 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-12-01T09:39:19.292 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2022-12-01T09:39:19.292 INFO:teuthology.orchestra.run.smithi002.stdout: chdir = "" 2022-12-01T09:39:19.292 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-12-01T09:39:19.293 INFO:teuthology.orchestra.run.smithi002.stdout: auth supported = cephx 2022-12-01T09:39:19.293 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.293 INFO:teuthology.orchestra.run.smithi002.stdout: filestore xattr use omap = true 2022-12-01T09:39:19.293 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.293 INFO:teuthology.orchestra.run.smithi002.stdout: mon clock drift allowed = 1.000 2022-12-01T09:39:19.294 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.294 INFO:teuthology.orchestra.run.smithi002.stdout: osd crush chooseleaf type = 0 2022-12-01T09:39:19.294 INFO:teuthology.orchestra.run.smithi002.stdout: auth debug = true 2022-12-01T09:39:19.294 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.294 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on old message = true 2022-12-01T09:39:19.295 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on bug = true 2022-12-01T09:39:19.295 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.295 INFO:teuthology.orchestra.run.smithi002.stdout: mon max pg per osd = 10000 # >= luminous 2022-12-01T09:39:19.295 INFO:teuthology.orchestra.run.smithi002.stdout: mon pg warn max object skew = 0 2022-12-01T09:39:19.296 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.296 INFO:teuthology.orchestra.run.smithi002.stdout: # disable pg_autoscaler by default for new pools 2022-12-01T09:39:19.296 INFO:teuthology.orchestra.run.smithi002.stdout: osd_pool_default_pg_autoscale_mode = off 2022-12-01T09:39:19.296 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.296 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default size = 2 2022-12-01T09:39:19.297 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.297 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow primary affinity = true 2022-12-01T09:39:19.297 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow pg remap = true 2022-12-01T09:39:19.297 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on legacy crush tunables = false 2022-12-01T09:39:19.298 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on crush straw calc version zero = false 2022-12-01T09:39:19.298 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on no sortbitwise = false 2022-12-01T09:39:19.298 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on osd down out interval zero = false 2022-12-01T09:39:19.298 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on too few osds = false 2022-12-01T09:39:19.299 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-12-01T09:39:19.299 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_no_redundancy = false 2022-12-01T09:39:19.300 INFO:teuthology.orchestra.run.smithi002.stdout: mon_allow_pool_size_one = true 2022-12-01T09:39:19.300 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.301 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-12-01T09:39:19.301 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.301 INFO:teuthology.orchestra.run.smithi002.stdout: osd default data pool replay window = 5 2022-12-01T09:39:19.302 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.302 INFO:teuthology.orchestra.run.smithi002.stdout: mon allow pool delete = true 2022-12-01T09:39:19.303 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.303 INFO:teuthology.orchestra.run.smithi002.stdout: mon cluster log file level = debug 2022-12-01T09:39:19.303 INFO:teuthology.orchestra.run.smithi002.stdout: debug asserts on shutdown = true 2022-12-01T09:39:19.304 INFO:teuthology.orchestra.run.smithi002.stdout: mon health detail to clog = false 2022-12-01T09:39:19.304 INFO:teuthology.orchestra.run.smithi002.stdout: mon host = "172.21.15.2,172.21.15.85,[v2:172.21.15.85:3301,v1:172.21.15.85:6790]" 2022-12-01T09:39:19.305 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on skipped message = False 2022-12-01T09:39:19.305 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = 3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803 2022-12-01T09:39:19.305 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.306 INFO:teuthology.orchestra.run.smithi002.stdout:[osd] 2022-12-01T09:39:19.306 INFO:teuthology.orchestra.run.smithi002.stdout: osd journal size = 100 2022-12-01T09:39:19.306 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.306 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub load threshold = 5.0 2022-12-01T09:39:19.307 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub max interval = 600 2022-12-01T09:39:19.307 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.307 INFO:teuthology.orchestra.run.smithi002.stdout: osd recover clone overlap = true 2022-12-01T09:39:19.307 INFO:teuthology.orchestra.run.smithi002.stdout: osd recovery max chunk = 1048576 2022-12-01T09:39:19.308 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.308 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug shutdown = true 2022-12-01T09:39:19.308 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug op order = true 2022-12-01T09:39:19.308 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug verify stray on activate = true 2022-12-01T09:39:19.308 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.309 INFO:teuthology.orchestra.run.smithi002.stdout: osd open classes on start = true 2022-12-01T09:39:19.309 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug pg log writeout = true 2022-12-01T09:39:19.309 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.309 INFO:teuthology.orchestra.run.smithi002.stdout: osd deep scrub update digest min age = 30 2022-12-01T09:39:19.310 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.310 INFO:teuthology.orchestra.run.smithi002.stdout: osd map max advance = 10 2022-12-01T09:39:19.310 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.310 INFO:teuthology.orchestra.run.smithi002.stdout: journal zero on create = true 2022-12-01T09:39:19.310 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.311 INFO:teuthology.orchestra.run.smithi002.stdout: filestore ondisk finisher threads = 3 2022-12-01T09:39:19.311 INFO:teuthology.orchestra.run.smithi002.stdout: filestore apply finisher threads = 3 2022-12-01T09:39:19.311 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.311 INFO:teuthology.orchestra.run.smithi002.stdout: bdev debug aio = true 2022-12-01T09:39:19.311 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug misdirected ops = true 2022-12-01T09:39:19.312 INFO:teuthology.orchestra.run.smithi002.stdout: bdev async discard = True 2022-12-01T09:39:19.312 INFO:teuthology.orchestra.run.smithi002.stdout: bdev enable discard = True 2022-12-01T09:39:19.312 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore allocator = bitmap 2022-12-01T09:39:19.312 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore block size = 96636764160 2022-12-01T09:39:19.313 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore fsck on mount = True 2022-12-01T09:39:19.313 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluefs = 1/20 2022-12-01T09:39:19.313 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluestore = 1/20 2022-12-01T09:39:19.313 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2022-12-01T09:39:19.313 INFO:teuthology.orchestra.run.smithi002.stdout: debug osd = 20 2022-12-01T09:39:19.314 INFO:teuthology.orchestra.run.smithi002.stdout: debug rocksdb = 4/10 2022-12-01T09:39:19.314 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd backfillfull_ratio = 0.85 2022-12-01T09:39:19.314 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd full ratio = 0.9 2022-12-01T09:39:19.314 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd nearfull ratio = 0.8 2022-12-01T09:39:19.314 INFO:teuthology.orchestra.run.smithi002.stdout: osd failsafe full ratio = 0.95 2022-12-01T09:39:19.315 INFO:teuthology.orchestra.run.smithi002.stdout: osd objectstore = bluestore 2022-12-01T09:39:19.315 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2022-12-01T09:39:19.315 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.315 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr] 2022-12-01T09:39:19.316 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2022-12-01T09:39:19.316 INFO:teuthology.orchestra.run.smithi002.stdout: debug mgr = 20 2022-12-01T09:39:19.316 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2022-12-01T09:39:19.316 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2022-12-01T09:39:19.316 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2022-12-01T09:39:19.317 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2022-12-01T09:39:19.317 INFO:teuthology.orchestra.run.smithi002.stdout: mgr/telemetry/nag = false 2022-12-01T09:39:19.317 INFO:teuthology.orchestra.run.smithi002.stdout: debug client = 20 2022-12-01T09:39:19.317 INFO:teuthology.orchestra.run.smithi002.stdout: debug finisher = 20 2022-12-01T09:39:19.318 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.318 INFO:teuthology.orchestra.run.smithi002.stdout:[mon] 2022-12-01T09:39:19.318 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2022-12-01T09:39:19.318 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2022-12-01T09:39:19.318 INFO:teuthology.orchestra.run.smithi002.stdout: debug paxos = 20 2022-12-01T09:39:19.319 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2022-12-01T09:39:19.319 INFO:teuthology.orchestra.run.smithi002.stdout: mon data avail warn = 5 2022-12-01T09:39:19.319 INFO:teuthology.orchestra.run.smithi002.stdout: mon mgr mkfs grace = 240 2022-12-01T09:39:19.319 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2022-12-01T09:39:19.319 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd reporter subtree level = osd 2022-12-01T09:39:19.320 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd prime pg temp = true 2022-12-01T09:39:19.320 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2022-12-01T09:39:19.320 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.320 INFO:teuthology.orchestra.run.smithi002.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-12-01T09:39:19.321 INFO:teuthology.orchestra.run.smithi002.stdout: auth mon ticket ttl = 660 # 11m 2022-12-01T09:39:19.321 INFO:teuthology.orchestra.run.smithi002.stdout: auth service ticket ttl = 240 # 4m 2022-12-01T09:39:19.321 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.321 INFO:teuthology.orchestra.run.smithi002.stdout: # don't complain about insecure global_id in the test suite 2022-12-01T09:39:19.321 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-12-01T09:39:19.322 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-12-01T09:39:19.322 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.322 INFO:teuthology.orchestra.run.smithi002.stdout: # 1m isn't quite enough 2022-12-01T09:39:19.322 INFO:teuthology.orchestra.run.smithi002.stdout: mon_down_mkfs_grace = 2m 2022-12-01T09:39:19.323 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.323 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_filestore_osds = false 2022-12-01T09:39:19.323 INFO:teuthology.orchestra.run.smithi002.stdout: mon op complaint time = 120 2022-12-01T09:39:19.323 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:39:19.324 INFO:teuthology.orchestra.run.smithi002.stdout:[client] 2022-12-01T09:39:19.324 INFO:teuthology.orchestra.run.smithi002.stdout: rgw cache enabled = true 2022-12-01T09:39:19.324 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable ops log = true 2022-12-01T09:39:19.324 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable usage log = true 2022-12-01T09:39:19.325 INFO:teuthology.orchestra.run.smithi002.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-12-01T09:39:19.325 INFO:teuthology.orchestra.run.smithi002.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-12-01T09:39:19.325 INFO:teuthology.orchestra.run.smithi002.stdout: client mount timeout = 600 2022-12-01T09:39:19.325 INFO:teuthology.orchestra.run.smithi002.stdout: debug client = 20 2022-12-01T09:39:19.326 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2022-12-01T09:39:19.326 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = "" 2022-12-01T09:39:19.326 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 900 2022-12-01T09:39:19.326 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 900 2022-12-01T09:39:19.326 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.a] 2022-12-01T09:39:19.327 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.b] 2022-12-01T09:39:19.327 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.c] 2022-12-01T09:39:19.327 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2022-12-01T09:39:19.327 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c] 2022-12-01T09:39:19.328 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2022-12-01T09:39:19.328 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d] 2022-12-01T09:39:19.328 INFO:teuthology.orchestra.run.smithi002.stdout:[mds] 2022-12-01T09:39:19.328 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds = 20 2022-12-01T09:39:19.328 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds balancer = 20 2022-12-01T09:39:19.329 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2022-12-01T09:39:19.329 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug frag = True 2022-12-01T09:39:19.329 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug scatterstat = True 2022-12-01T09:39:19.329 INFO:teuthology.orchestra.run.smithi002.stdout: mds op complaint time = 180 2022-12-01T09:39:19.330 INFO:teuthology.orchestra.run.smithi002.stdout: mds verify scatter = True 2022-12-01T09:39:19.330 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2022-12-01T09:39:19.330 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 900 2022-12-01T09:39:19.330 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 900 2022-12-01T09:39:19.333 INFO:teuthology.orchestra.run.smithi085.stdout:[global] 2022-12-01T09:39:19.333 INFO:teuthology.orchestra.run.smithi085.stdout: chdir = "" 2022-12-01T09:39:19.333 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-12-01T09:39:19.333 INFO:teuthology.orchestra.run.smithi085.stdout: auth supported = cephx 2022-12-01T09:39:19.334 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.334 INFO:teuthology.orchestra.run.smithi085.stdout: filestore xattr use omap = true 2022-12-01T09:39:19.334 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.334 INFO:teuthology.orchestra.run.smithi085.stdout: mon clock drift allowed = 1.000 2022-12-01T09:39:19.335 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.335 INFO:teuthology.orchestra.run.smithi085.stdout: osd crush chooseleaf type = 0 2022-12-01T09:39:19.335 INFO:teuthology.orchestra.run.smithi085.stdout: auth debug = true 2022-12-01T09:39:19.335 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.336 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on old message = true 2022-12-01T09:39:19.336 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on bug = true 2022-12-01T09:39:19.336 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: mon max pg per osd = 10000 # >= luminous 2022-12-01T09:39:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: mon pg warn max object skew = 0 2022-12-01T09:39:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: # disable pg_autoscaler by default for new pools 2022-12-01T09:39:19.337 INFO:teuthology.orchestra.run.smithi085.stdout: osd_pool_default_pg_autoscale_mode = off 2022-12-01T09:39:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default size = 2 2022-12-01T09:39:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.338 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow primary affinity = true 2022-12-01T09:39:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd allow pg remap = true 2022-12-01T09:39:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on legacy crush tunables = false 2022-12-01T09:39:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on crush straw calc version zero = false 2022-12-01T09:39:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on no sortbitwise = false 2022-12-01T09:39:19.339 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on osd down out interval zero = false 2022-12-01T09:39:19.340 INFO:teuthology.orchestra.run.smithi085.stdout: mon warn on too few osds = false 2022-12-01T09:39:19.340 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-12-01T09:39:19.340 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_pool_no_redundancy = false 2022-12-01T09:39:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: mon_allow_pool_size_one = true 2022-12-01T09:39:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.341 INFO:teuthology.orchestra.run.smithi085.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-12-01T09:39:19.342 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.342 INFO:teuthology.orchestra.run.smithi085.stdout: osd default data pool replay window = 5 2022-12-01T09:39:19.342 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: mon allow pool delete = true 2022-12-01T09:39:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: mon cluster log file level = debug 2022-12-01T09:39:19.343 INFO:teuthology.orchestra.run.smithi085.stdout: debug asserts on shutdown = true 2022-12-01T09:39:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: mon health detail to clog = false 2022-12-01T09:39:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: mon host = "172.21.15.2,172.21.15.85,[v2:172.21.15.85:3301,v1:172.21.15.85:6790]" 2022-12-01T09:39:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: ms die on skipped message = False 2022-12-01T09:39:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: fsid = 3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803 2022-12-01T09:39:19.344 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.345 INFO:teuthology.orchestra.run.smithi085.stdout:[osd] 2022-12-01T09:39:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: osd journal size = 100 2022-12-01T09:39:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.345 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub load threshold = 5.0 2022-12-01T09:39:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: osd scrub max interval = 600 2022-12-01T09:39:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: osd recover clone overlap = true 2022-12-01T09:39:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: osd recovery max chunk = 1048576 2022-12-01T09:39:19.346 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.347 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug shutdown = true 2022-12-01T09:39:19.347 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug op order = true 2022-12-01T09:39:19.347 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug verify stray on activate = true 2022-12-01T09:39:19.347 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: osd open classes on start = true 2022-12-01T09:39:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug pg log writeout = true 2022-12-01T09:39:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.348 INFO:teuthology.orchestra.run.smithi085.stdout: osd deep scrub update digest min age = 30 2022-12-01T09:39:19.349 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.349 INFO:teuthology.orchestra.run.smithi085.stdout: osd map max advance = 10 2022-12-01T09:39:19.350 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.350 INFO:teuthology.orchestra.run.smithi085.stdout: journal zero on create = true 2022-12-01T09:39:19.350 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.351 INFO:teuthology.orchestra.run.smithi085.stdout: filestore ondisk finisher threads = 3 2022-12-01T09:39:19.351 INFO:teuthology.orchestra.run.smithi085.stdout: filestore apply finisher threads = 3 2022-12-01T09:39:19.352 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.352 INFO:teuthology.orchestra.run.smithi085.stdout: bdev debug aio = true 2022-12-01T09:39:19.353 INFO:teuthology.orchestra.run.smithi085.stdout: osd debug misdirected ops = true 2022-12-01T09:39:19.353 INFO:teuthology.orchestra.run.smithi085.stdout: bdev async discard = True 2022-12-01T09:39:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: bdev enable discard = True 2022-12-01T09:39:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore allocator = bitmap 2022-12-01T09:39:19.354 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore block size = 96636764160 2022-12-01T09:39:19.355 INFO:teuthology.orchestra.run.smithi085.stdout: bluestore fsck on mount = True 2022-12-01T09:39:19.355 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluefs = 1/20 2022-12-01T09:39:19.356 INFO:teuthology.orchestra.run.smithi085.stdout: debug bluestore = 1/20 2022-12-01T09:39:19.356 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-12-01T09:39:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: debug osd = 20 2022-12-01T09:39:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: debug rocksdb = 4/10 2022-12-01T09:39:19.357 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd backfillfull_ratio = 0.85 2022-12-01T09:39:19.358 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd full ratio = 0.9 2022-12-01T09:39:19.358 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd nearfull ratio = 0.8 2022-12-01T09:39:19.358 INFO:teuthology.orchestra.run.smithi085.stdout: osd failsafe full ratio = 0.95 2022-12-01T09:39:19.359 INFO:teuthology.orchestra.run.smithi085.stdout: osd objectstore = bluestore 2022-12-01T09:39:19.359 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-12-01T09:39:19.360 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.360 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr] 2022-12-01T09:39:19.360 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-12-01T09:39:19.360 INFO:teuthology.orchestra.run.smithi085.stdout: debug mgr = 20 2022-12-01T09:39:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-12-01T09:39:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-12-01T09:39:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-12-01T09:39:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-12-01T09:39:19.361 INFO:teuthology.orchestra.run.smithi085.stdout: mgr/telemetry/nag = false 2022-12-01T09:39:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: debug client = 20 2022-12-01T09:39:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: debug finisher = 20 2022-12-01T09:39:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.362 INFO:teuthology.orchestra.run.smithi085.stdout:[mon] 2022-12-01T09:39:19.362 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-12-01T09:39:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: debug mon = 20 2022-12-01T09:39:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: debug paxos = 20 2022-12-01T09:39:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: debug auth = 20 2022-12-01T09:39:19.363 INFO:teuthology.orchestra.run.smithi085.stdout: mon data avail warn = 5 2022-12-01T09:39:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: mon mgr mkfs grace = 240 2022-12-01T09:39:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min pgs per osd = 4 2022-12-01T09:39:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd reporter subtree level = osd 2022-12-01T09:39:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: mon osd prime pg temp = true 2022-12-01T09:39:19.364 INFO:teuthology.orchestra.run.smithi085.stdout: mon reweight min bytes per osd = 10 2022-12-01T09:39:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-12-01T09:39:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: auth mon ticket ttl = 660 # 11m 2022-12-01T09:39:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: auth service ticket ttl = 240 # 4m 2022-12-01T09:39:19.365 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.366 INFO:teuthology.orchestra.run.smithi085.stdout: # don't complain about insecure global_id in the test suite 2022-12-01T09:39:19.366 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-12-01T09:39:19.366 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-12-01T09:39:19.366 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.367 INFO:teuthology.orchestra.run.smithi085.stdout: # 1m isn't quite enough 2022-12-01T09:39:19.367 INFO:teuthology.orchestra.run.smithi085.stdout: mon_down_mkfs_grace = 2m 2022-12-01T09:39:19.367 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.367 INFO:teuthology.orchestra.run.smithi085.stdout: mon_warn_on_filestore_osds = false 2022-12-01T09:39:19.367 INFO:teuthology.orchestra.run.smithi085.stdout: mon op complaint time = 120 2022-12-01T09:39:19.368 INFO:teuthology.orchestra.run.smithi085.stdout: 2022-12-01T09:39:19.368 INFO:teuthology.orchestra.run.smithi085.stdout:[client] 2022-12-01T09:39:19.368 INFO:teuthology.orchestra.run.smithi085.stdout: rgw cache enabled = true 2022-12-01T09:39:19.368 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable ops log = true 2022-12-01T09:39:19.368 INFO:teuthology.orchestra.run.smithi085.stdout: rgw enable usage log = true 2022-12-01T09:39:19.369 INFO:teuthology.orchestra.run.smithi085.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-12-01T09:39:19.369 INFO:teuthology.orchestra.run.smithi085.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-12-01T09:39:19.369 INFO:teuthology.orchestra.run.smithi085.stdout: client mount timeout = 600 2022-12-01T09:39:19.369 INFO:teuthology.orchestra.run.smithi085.stdout: debug client = 20 2022-12-01T09:39:19.370 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-12-01T09:39:19.370 INFO:teuthology.orchestra.run.smithi085.stdout: pid file = "" 2022-12-01T09:39:19.370 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 900 2022-12-01T09:39:19.370 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 900 2022-12-01T09:39:19.370 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.a] 2022-12-01T09:39:19.371 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.b] 2022-12-01T09:39:19.371 INFO:teuthology.orchestra.run.smithi085.stdout:[mon.c] 2022-12-01T09:39:19.371 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2022-12-01T09:39:19.371 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.c] 2022-12-01T09:39:19.372 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2022-12-01T09:39:19.372 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.d] 2022-12-01T09:39:19.372 INFO:teuthology.orchestra.run.smithi085.stdout:[mds] 2022-12-01T09:39:19.372 INFO:teuthology.orchestra.run.smithi085.stdout: debug mds = 20 2022-12-01T09:39:19.373 INFO:teuthology.orchestra.run.smithi085.stdout: debug mds balancer = 20 2022-12-01T09:39:19.373 INFO:teuthology.orchestra.run.smithi085.stdout: debug ms = 1 2022-12-01T09:39:19.373 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug frag = True 2022-12-01T09:39:19.373 INFO:teuthology.orchestra.run.smithi085.stdout: mds debug scatterstat = True 2022-12-01T09:39:19.373 INFO:teuthology.orchestra.run.smithi085.stdout: mds op complaint time = 180 2022-12-01T09:39:19.374 INFO:teuthology.orchestra.run.smithi085.stdout: mds verify scatter = True 2022-12-01T09:39:19.374 INFO:teuthology.orchestra.run.smithi085.stdout: osd op complaint time = 180 2022-12-01T09:39:19.374 INFO:teuthology.orchestra.run.smithi085.stdout: rados mon op timeout = 900 2022-12-01T09:39:19.374 INFO:teuthology.orchestra.run.smithi085.stdout: rados osd op timeout = 900 2022-12-01T09:39:19.375 INFO:tasks.ceph:Creating admin key on mon.a... 2022-12-01T09:39:19.375 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-12-01T09:39:19.434 INFO:tasks.ceph:Copying monmap to all nodes... 2022-12-01T09:39:19.434 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:19.435 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-12-01T09:39:19.452 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:19.452 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-12-01T09:39:19.508 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi002.front.sepia.ceph.com 2022-12-01T09:39:19.509 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:19.509 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.keyring 2022-12-01T09:39:19.510 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-12-01T09:39:19.587 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:19.587 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:19.644 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi085.front.sepia.ceph.com 2022-12-01T09:39:19.645 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:19.645 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd of=/etc/ceph/ceph.keyring 2022-12-01T09:39:19.645 DEBUG:teuthology.orchestra.run.smithi085:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-12-01T09:39:19.684 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:19.685 DEBUG:teuthology.orchestra.run.smithi085:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:19.741 INFO:tasks.ceph:Setting up mon nodes... 2022-12-01T09:39:19.741 INFO:tasks.ceph:Setting up mgr nodes... 2022-12-01T09:39:19.742 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-12-01T09:39:19.797 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-12-01T09:39:19.801 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-12-01T09:39:19.858 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-12-01T09:39:19.861 INFO:tasks.ceph:Setting up mds nodes... 2022-12-01T09:39:19.861 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-12-01T09:39:19.917 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-12-01T09:39:19.919 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-12-01T09:39:19.947 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-12-01T09:39:20.043 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-12-01T09:39:20.046 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-12-01T09:39:20.073 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-12-01T09:39:20.127 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-12-01T09:39:20.130 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-12-01T09:39:20.157 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-12-01T09:39:20.249 INFO:teuthology.orchestra.run.smithi085.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-12-01T09:39:20.252 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-12-01T09:39:20.278 INFO:tasks.ceph_client:Setting up client nodes... 2022-12-01T09:39:20.279 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-12-01T09:39:20.324 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-12-01T09:39:20.339 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-12-01T09:39:20.383 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-12-01T09:39:20.397 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-12-01T09:39:20.441 INFO:teuthology.orchestra.run.smithi085.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-12-01T09:39:20.453 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2022-12-01T09:39:20.537 INFO:teuthology.orchestra.run.smithi085.stdout:creating /etc/ceph/ceph.client.3.keyring 2022-12-01T09:39:20.549 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-12-01T09:39:20.550 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi085.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-12-01T09:39:20.550 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-12-01T09:39:20.579 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:20.579 INFO:tasks.ceph:role: osd.0 2022-12-01T09:39:20.579 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com 2022-12-01T09:39:20.580 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-12-01T09:39:20.646 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:39:20.647 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:39:20.647 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:39:20.647 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:39:20.648 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:39:20.648 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:39:20.648 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:39:20.648 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:39:20.648 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:39:20.649 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:39:20.669 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2022-12-01T09:39:20.670 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2022-12-01T09:39:20.671 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-12-01T09:39:20.752 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-12-01T09:39:20.825 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-12-01T09:39:20.893 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:20.894 INFO:tasks.ceph:role: osd.1 2022-12-01T09:39:20.894 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com 2022-12-01T09:39:20.895 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-12-01T09:39:20.960 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:39:20.960 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:39:20.961 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:39:20.961 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:39:20.962 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:39:20.962 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:39:20.963 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:39:20.963 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:39:20.963 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:39:20.964 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:39:20.982 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2022-12-01T09:39:20.984 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2022-12-01T09:39:20.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-12-01T09:39:21.067 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-12-01T09:39:21.139 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-12-01T09:39:21.206 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:21.207 INFO:tasks.ceph:role: osd.2 2022-12-01T09:39:21.207 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com 2022-12-01T09:39:21.208 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-12-01T09:39:21.274 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:39:21.275 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:39:21.275 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:39:21.275 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:39:21.276 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:39:21.276 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:39:21.276 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:39:21.276 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:39:21.277 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:39:21.277 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:39:21.296 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2022-12-01T09:39:21.298 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2022-12-01T09:39:21.299 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-12-01T09:39:21.379 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-12-01T09:39:21.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-12-01T09:39:21.517 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:21.518 INFO:tasks.ceph:role: osd.3 2022-12-01T09:39:21.518 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com 2022-12-01T09:39:21.519 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-12-01T09:39:21.585 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:39:21.585 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:39:21.585 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:39:21.586 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:39:21.586 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:39:21.586 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:39:21.586 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:39:21.586 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:39:21.587 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:39:21.587 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:39:21.608 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2022-12-01T09:39:21.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2022-12-01T09:39:21.610 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-12-01T09:39:21.690 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-12-01T09:39:21.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:21.850 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:21.847+0000 7f6244a0a5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-12-01T09:39:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:21.847+0000 7f6244a0a5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-12-01T09:39:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:21.847+0000 7f6244a0a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-12-01T09:39:21.851 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:21.847+0000 7f6244a0a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-12-01T09:39:21.852 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:21.847+0000 7f6244a0a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-12-01T09:39:21.852 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:21.847+0000 7f6244a0a5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-12-01T09:39:24.715 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-12-01T09:39:24.748 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:24.842 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:24.838+0000 7ff5561695c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-12-01T09:39:24.843 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:24.839+0000 7ff5561695c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-12-01T09:39:24.843 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:24.839+0000 7ff5561695c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-12-01T09:39:24.843 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:24.839+0000 7ff5561695c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-12-01T09:39:24.843 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:24.839+0000 7ff5561695c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-12-01T09:39:24.844 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:24.839+0000 7ff5561695c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-12-01T09:39:27.705 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-12-01T09:39:27.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:27.832 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:27.829+0000 7f4536b255c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-12-01T09:39:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:27.829+0000 7f4536b255c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-12-01T09:39:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:27.829+0000 7f4536b255c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-12-01T09:39:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:27.829+0000 7f4536b255c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-12-01T09:39:27.833 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:27.829+0000 7f4536b255c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-12-01T09:39:27.834 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:27.829+0000 7f4536b255c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-12-01T09:39:30.698 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-12-01T09:39:30.733 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:30.829 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:30.825+0000 7f427c6915c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-12-01T09:39:30.830 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:30.826+0000 7f427c6915c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-12-01T09:39:30.830 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:30.826+0000 7f427c6915c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-12-01T09:39:30.830 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:30.826+0000 7f427c6915c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-12-01T09:39:30.831 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:30.826+0000 7f427c6915c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-12-01T09:39:30.831 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:30.826+0000 7f427c6915c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-12-01T09:39:33.694 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-12-01T09:39:33.727 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-12-01T09:39:33.762 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:33.763 INFO:tasks.ceph:role: osd.4 2022-12-01T09:39:33.763 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi085.front.sepia.ceph.com 2022-12-01T09:39:33.763 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-12-01T09:39:33.839 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:39:33.839 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:39:33.840 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:39:33.840 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:39:33.840 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:39:33.840 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:39:33.840 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:39:33.841 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:39:33.841 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:39:33.841 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:39:34.278 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2022-12-01T09:39:34.280 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2022-12-01T09:39:34.280 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-12-01T09:39:34.464 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-12-01T09:39:34.505 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-12-01T09:39:34.575 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:34.576 INFO:tasks.ceph:role: osd.5 2022-12-01T09:39:34.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi085.front.sepia.ceph.com 2022-12-01T09:39:34.577 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-12-01T09:39:34.653 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:39:34.653 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:39:34.653 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:39:34.653 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:39:34.654 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:39:34.654 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:39:34.654 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:39:34.654 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:39:34.655 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:39:34.655 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:39:35.067 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2022-12-01T09:39:35.068 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2022-12-01T09:39:35.069 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-12-01T09:39:35.179 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-12-01T09:39:35.215 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-12-01T09:39:35.284 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:35.285 INFO:tasks.ceph:role: osd.6 2022-12-01T09:39:35.285 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi085.front.sepia.ceph.com 2022-12-01T09:39:35.286 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-12-01T09:39:35.360 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:39:35.361 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:39:35.361 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:39:35.361 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:39:35.361 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:39:35.362 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:39:35.362 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:39:35.362 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:39:35.362 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:39:35.363 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:39:35.802 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2022-12-01T09:39:35.804 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2022-12-01T09:39:35.804 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-12-01T09:39:35.957 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-12-01T09:39:35.993 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-12-01T09:39:36.063 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:39:36.064 INFO:tasks.ceph:role: osd.7 2022-12-01T09:39:36.064 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi085.front.sepia.ceph.com 2022-12-01T09:39:36.065 DEBUG:teuthology.orchestra.run.smithi085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-12-01T09:39:36.141 INFO:teuthology.orchestra.run.smithi085.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:39:36.141 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:39:36.141 INFO:teuthology.orchestra.run.smithi085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:39:36.141 INFO:teuthology.orchestra.run.smithi085.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:39:36.142 INFO:teuthology.orchestra.run.smithi085.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:39:36.142 INFO:teuthology.orchestra.run.smithi085.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:39:36.142 INFO:teuthology.orchestra.run.smithi085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:39:36.142 INFO:teuthology.orchestra.run.smithi085.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:39:36.142 INFO:teuthology.orchestra.run.smithi085.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:39:36.143 INFO:teuthology.orchestra.run.smithi085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:39:36.584 INFO:teuthology.orchestra.run.smithi085.stdout:Discarding blocks...Done. 2022-12-01T09:39:36.586 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi085.front.sepia.ceph.com -o noatime 2022-12-01T09:39:36.586 DEBUG:teuthology.orchestra.run.smithi085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-12-01T09:39:36.769 DEBUG:teuthology.orchestra.run.smithi085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-12-01T09:39:36.806 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:36.899 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:36.895+0000 7f80c05f45c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-12-01T09:39:36.900 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:36.895+0000 7f80c05f45c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-12-01T09:39:36.900 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:36.895+0000 7f80c05f45c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-12-01T09:39:36.900 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:36.895+0000 7f80c05f45c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-12-01T09:39:36.900 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:36.895+0000 7f80c05f45c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-12-01T09:39:36.900 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:36.897+0000 7f80c05f45c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-12-01T09:39:39.757 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-12-01T09:39:39.789 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:39.880 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:39.876+0000 7f5e707855c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-12-01T09:39:39.881 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:39.876+0000 7f5e707855c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-12-01T09:39:39.881 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:39.876+0000 7f5e707855c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-12-01T09:39:39.881 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:39.876+0000 7f5e707855c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-12-01T09:39:39.882 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:39.876+0000 7f5e707855c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-12-01T09:39:39.882 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:39.877+0000 7f5e707855c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-12-01T09:39:42.749 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-12-01T09:39:42.781 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:42.873 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:42.869+0000 7f30eda7e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-12-01T09:39:42.873 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:42.869+0000 7f30eda7e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-12-01T09:39:42.874 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:42.869+0000 7f30eda7e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-12-01T09:39:42.874 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:42.869+0000 7f30eda7e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-12-01T09:39:42.874 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:42.869+0000 7f30eda7e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-12-01T09:39:42.875 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:42.870+0000 7f30eda7e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-12-01T09:39:45.728 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-12-01T09:39:45.760 DEBUG:teuthology.orchestra.run.smithi085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:45.851 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:45.848+0000 7fc8672425c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-12-01T09:39:45.852 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:45.848+0000 7fc8672425c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-12-01T09:39:45.852 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:45.848+0000 7fc8672425c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-12-01T09:39:45.852 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:45.848+0000 7fc8672425c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-12-01T09:39:45.853 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:45.848+0000 7fc8672425c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-12-01T09:39:45.853 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:45.849+0000 7fc8672425c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-12-01T09:39:47.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-12-01T09:39:47.190 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-12-01T09:39:48.706 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-12-01T09:39:48.738 INFO:tasks.ceph:Reading keys from all nodes... 2022-12-01T09:39:48.738 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:48.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-12-01T09:39:48.772 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:48.773 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-12-01T09:39:48.845 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:48.845 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-12-01T09:39:48.916 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:48.916 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-12-01T09:39:48.986 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:48.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-12-01T09:39:49.056 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:49.057 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-12-01T09:39:49.126 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:49.127 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-12-01T09:39:49.196 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.197 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-12-01T09:39:49.229 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.229 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-12-01T09:39:49.299 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.300 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-12-01T09:39:49.368 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.368 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-12-01T09:39:49.436 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.436 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-12-01T09:39:49.503 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.503 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-12-01T09:39:49.571 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.571 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-12-01T09:39:49.639 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:49.640 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-12-01T09:39:49.661 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:49.661 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2022-12-01T09:39:49.722 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.722 DEBUG:teuthology.orchestra.run.smithi085:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-12-01T09:39:49.741 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:49.741 DEBUG:teuthology.orchestra.run.smithi085:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2022-12-01T09:39:49.798 INFO:tasks.ceph:Adding keys to all mons... 2022-12-01T09:39:49.799 DEBUG:teuthology.orchestra.run.smithi002:> sudo tee -a /etc/ceph/ceph.keyring 2022-12-01T09:39:49.802 DEBUG:teuthology.orchestra.run.smithi085:> sudo tee -a /etc/ceph/ceph.keyring 2022-12-01T09:39:49.841 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.y] 2022-12-01T09:39:49.842 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBHdohjxyRqLxAAkoNs6BahPr5+G9kcDmK8OA== 2022-12-01T09:39:49.842 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2022-12-01T09:39:49.842 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBHdohjLh2KNhAAPohm90APLF4CQqPl5EyVNA== 2022-12-01T09:39:49.842 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c] 2022-12-01T09:39:49.843 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBIdohjZ3l/AhAACJ8gPY7POq3QGxAaeTzcHA== 2022-12-01T09:39:49.843 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.0] 2022-12-01T09:39:49.844 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBJdohjSnKQMhAAyeBe65aIf/x39cqK9JmoMQ== 2022-12-01T09:39:49.844 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.1] 2022-12-01T09:39:49.844 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBMdohjdW8QMhAAFA9FVUiCMQRuaVXTuEj4WQ== 2022-12-01T09:39:49.844 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.2] 2022-12-01T09:39:49.845 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBPdohjAuR4MRAAjGI5N+LQ9KW9PmQQdrguCw== 2022-12-01T09:39:49.845 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.3] 2022-12-01T09:39:49.845 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBSdohj9ppKMRAAikeeWIlkku6LKJagvHIFug== 2022-12-01T09:39:49.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.x] 2022-12-01T09:39:49.846 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBHdohjM8cPMxAAN5vtWdqs/X3F5IOSsfYJGA== 2022-12-01T09:39:49.846 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2022-12-01T09:39:49.846 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBIdohj0MF4BxAAgwm7K1dqS9qJ8A/VmrQKAA== 2022-12-01T09:39:49.846 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d] 2022-12-01T09:39:49.847 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBIdohj5+/GDhAA1lljIa3e+0UW0xST2eKCLg== 2022-12-01T09:39:49.847 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.4] 2022-12-01T09:39:49.847 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBYdohj5D1zNRAAgIG3nlQv6PBO/zBNpTyngA== 2022-12-01T09:39:49.847 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.5] 2022-12-01T09:39:49.848 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBbdohjPnJPNBAAjkuiMberz+K3MZZnkX74bQ== 2022-12-01T09:39:49.848 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.6] 2022-12-01T09:39:49.848 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBedohjH/LfMxAAwlqLsPznITXnzu15aQsHPQ== 2022-12-01T09:39:49.848 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.7] 2022-12-01T09:39:49.848 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBhdohjFfycMhAAJGXprlIRvkxRQxc+c9ptCA== 2022-12-01T09:39:49.849 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2022-12-01T09:39:49.849 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBIdohjzlQ/ExAAogwuoRK57lGTR6/ohatbRQ== 2022-12-01T09:39:49.849 INFO:teuthology.orchestra.run.smithi002.stdout:[client.2] 2022-12-01T09:39:49.849 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBIdohj2eG/FhAAAsn69x6YPnr2vkkU7GinPg== 2022-12-01T09:39:49.850 INFO:teuthology.orchestra.run.smithi002.stdout:[client.1] 2022-12-01T09:39:49.850 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBIdohjodwpGhAAYKUU8ejKJB/1MpOpagd+wQ== 2022-12-01T09:39:49.850 INFO:teuthology.orchestra.run.smithi002.stdout:[client.3] 2022-12-01T09:39:49.850 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBIdohjWC7mHxAAijjRpAZ8UrRTiaprdBnfnw== 2022-12-01T09:39:49.864 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr.y] 2022-12-01T09:39:49.864 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBHdohjxyRqLxAAkoNs6BahPr5+G9kcDmK8OA== 2022-12-01T09:39:49.864 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.a] 2022-12-01T09:39:49.865 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBHdohjLh2KNhAAPohm90APLF4CQqPl5EyVNA== 2022-12-01T09:39:49.865 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.c] 2022-12-01T09:39:49.865 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBIdohjZ3l/AhAACJ8gPY7POq3QGxAaeTzcHA== 2022-12-01T09:39:49.865 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.0] 2022-12-01T09:39:49.866 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBJdohjSnKQMhAAyeBe65aIf/x39cqK9JmoMQ== 2022-12-01T09:39:49.866 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.1] 2022-12-01T09:39:49.866 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBMdohjdW8QMhAAFA9FVUiCMQRuaVXTuEj4WQ== 2022-12-01T09:39:49.866 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.2] 2022-12-01T09:39:49.866 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBPdohjAuR4MRAAjGI5N+LQ9KW9PmQQdrguCw== 2022-12-01T09:39:49.867 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.3] 2022-12-01T09:39:49.867 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBSdohj9ppKMRAAikeeWIlkku6LKJagvHIFug== 2022-12-01T09:39:49.867 INFO:teuthology.orchestra.run.smithi085.stdout:[mgr.x] 2022-12-01T09:39:49.868 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBHdohjM8cPMxAAN5vtWdqs/X3F5IOSsfYJGA== 2022-12-01T09:39:49.868 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.b] 2022-12-01T09:39:49.868 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBIdohj0MF4BxAAgwm7K1dqS9qJ8A/VmrQKAA== 2022-12-01T09:39:49.868 INFO:teuthology.orchestra.run.smithi085.stdout:[mds.d] 2022-12-01T09:39:49.869 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBIdohj5+/GDhAA1lljIa3e+0UW0xST2eKCLg== 2022-12-01T09:39:49.869 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.4] 2022-12-01T09:39:49.869 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBYdohj5D1zNRAAgIG3nlQv6PBO/zBNpTyngA== 2022-12-01T09:39:49.869 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.5] 2022-12-01T09:39:49.870 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBbdohjPnJPNBAAjkuiMberz+K3MZZnkX74bQ== 2022-12-01T09:39:49.870 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.6] 2022-12-01T09:39:49.870 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBedohjH/LfMxAAwlqLsPznITXnzu15aQsHPQ== 2022-12-01T09:39:49.870 INFO:teuthology.orchestra.run.smithi085.stdout:[osd.7] 2022-12-01T09:39:49.871 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBhdohjFfycMhAAJGXprlIRvkxRQxc+c9ptCA== 2022-12-01T09:39:49.871 INFO:teuthology.orchestra.run.smithi085.stdout:[client.0] 2022-12-01T09:39:49.871 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBIdohjzlQ/ExAAogwuoRK57lGTR6/ohatbRQ== 2022-12-01T09:39:49.871 INFO:teuthology.orchestra.run.smithi085.stdout:[client.2] 2022-12-01T09:39:49.872 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBIdohj2eG/FhAAAsn69x6YPnr2vkkU7GinPg== 2022-12-01T09:39:49.872 INFO:teuthology.orchestra.run.smithi085.stdout:[client.1] 2022-12-01T09:39:49.872 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBIdohjodwpGhAAYKUU8ejKJB/1MpOpagd+wQ== 2022-12-01T09:39:49.872 INFO:teuthology.orchestra.run.smithi085.stdout:[client.3] 2022-12-01T09:39:49.873 INFO:teuthology.orchestra.run.smithi085.stdout: key = AQBIdohjWC7mHxAAijjRpAZ8UrRTiaprdBnfnw== 2022-12-01T09:39:49.874 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-12-01T09:39:49.895 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-12-01T09:39:49.967 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:39:49.970 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:39:50.028 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:39:50.030 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:39:50.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.090 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.145 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.201 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.257 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.309 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-12-01T09:39:50.313 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-12-01T09:39:50.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:39:50.369 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:39:50.425 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:39:50.427 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:39:50.482 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.485 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.542 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.596 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.599 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.652 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.655 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:39:50.707 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:39:50.711 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:39:50.767 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:39:50.769 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:39:50.825 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:39:50.828 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:39:50.890 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:39:50.893 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:39:50.950 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-12-01T09:39:50.950 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-12-01T09:39:50.978 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-12-01T09:39:51.076 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-12-01T09:39:51.104 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-12-01T09:39:51.132 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-12-01T09:39:51.229 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-12-01T09:39:51.255 DEBUG:teuthology.orchestra.run.smithi085:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-12-01T09:39:51.323 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-12-01T09:39:51.419 DEBUG:teuthology.orchestra.run.smithi085:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-12-01T09:39:51.446 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:51.449 DEBUG:teuthology.orchestra.run.smithi085:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:39:51.503 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-12-01T09:39:51.503 INFO:tasks.ceph.mon.a:Restarting daemon 2022-12-01T09:39:51.504 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-12-01T09:39:51.508 INFO:tasks.ceph.mon.a:Started 2022-12-01T09:39:51.508 INFO:tasks.ceph.mon.b:Restarting daemon 2022-12-01T09:39:51.509 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-12-01T09:39:51.546 INFO:tasks.ceph.mon.b:Started 2022-12-01T09:39:51.547 INFO:tasks.ceph.mon.c:Restarting daemon 2022-12-01T09:39:51.547 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-12-01T09:39:51.549 INFO:tasks.ceph.mon.c:Started 2022-12-01T09:39:51.550 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-12-01T09:39:51.550 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-12-01T09:39:51.551 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-12-01T09:39:51.553 INFO:tasks.ceph.mgr.y:Started 2022-12-01T09:39:51.554 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-12-01T09:39:51.554 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-12-01T09:39:51.557 INFO:tasks.ceph.mgr.x:Started 2022-12-01T09:39:51.557 INFO:tasks.ceph:Setting crush tunables to default 2022-12-01T09:39:51.557 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd crush tunables default 2022-12-01T09:39:51.683 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.680+0000 7fa1205b0700 1 Processor -- start 2022-12-01T09:39:51.684 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.680+0000 7fa1205b0700 1 -- start start 2022-12-01T09:39:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa1205b0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:51.685 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa1205b0700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1180ce740 con 0x7fa11806d3f0 2022-12-01T09:39:51.686 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa1205b0700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11806bf80 con 0x7fa11806fae0 2022-12-01T09:39:51.686 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa1205b0700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180c9650 con 0x7fa1180ca0c0 2022-12-01T09:39:51.686 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa11db4b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa11806d3f0 0x7fa1180c8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:33694/0 (socket says 172.21.15.2:33694) 2022-12-01T09:39:51.686 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 learned_addr learned my addr 172.21.15.2:0/863644071 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:39:51.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:51.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:51.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:51.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:51.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.681+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:39:51.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.882+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:51.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.882+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:51.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.882+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:51.887 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.882+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:51.887 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:51.882+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-12-01T09:39:52.287 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:52.283+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:52.287 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:52.283+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:52.287 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:52.283+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:52.287 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:52.283+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:52.288 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:52.283+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-12-01T09:39:53.088 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:53.084+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:53.088 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:53.085+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:53.088 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:53.085+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:53.085+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:53.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:53.085+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-12-01T09:39:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 msgr2=0x7fa1180cdaa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:39:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa117fff700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa1180ca0c0 0x7fa1180cdaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 >> v1:172.21.15.85:6789/0 conn(0x7fa11806fae0 legacy=0x7fa11806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 >> v1:172.21.15.2:6789/0 conn(0x7fa11806d3f0 legacy=0x7fa1180c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa117fff700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa117fff700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1180c9650 con 0x7fa0f8006260 2022-12-01T09:39:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f8008db0 con 0x7fa0f80037a0 2022-12-01T09:39:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:54.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:39:54.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:54.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:54.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:39:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.681+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.682+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.682+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.682+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.682+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:39:54.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.882+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:54.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.882+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.882+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.882+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:54.887 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.882+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-12-01T09:39:54.888 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.883+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:54.889 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.883+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:54.889 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.883+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:54.889 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.883+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:54.889 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:54.883+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-12-01T09:39:55.287 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.283+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:55.287 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.283+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:55.287 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.283+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:55.287 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.283+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:55.288 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.283+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-12-01T09:39:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.284+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.284+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.284+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.284+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:55.290 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:55.284+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-12-01T09:39:56.088 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.084+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:56.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.084+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:56.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.084+0000 7fa11db4b700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:56.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.084+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:56.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.084+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-12-01T09:39:56.090 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.085+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:56.090 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.085+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:39:56.090 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.085+0000 7fa11e34c700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:39:56.090 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.085+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:39:56.091 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:56.085+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-12-01T09:39:57.018 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.015+0000 7feafdd4a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-12-01T09:39:57.024 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:57.021+0000 7f73f316d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-12-01T09:39:57.082 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.078+0000 7feafdd4a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-12-01T09:39:57.091 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:57.088+0000 7f73f316d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-12-01T09:39:57.155 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.152+0000 7feafdd4a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-12-01T09:39:57.168 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:57.165+0000 7f73f316d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-12-01T09:39:57.232 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.229+0000 7feafdd4a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-12-01T09:39:57.249 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:57.246+0000 7f73f316d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-12-01T09:39:57.371 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.367+0000 7feafdd4a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-12-01T09:39:57.394 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:57.391+0000 7f73f316d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-12-01T09:39:57.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 msgr2=0x7fa0f8005c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:39:57.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f80037a0 0x7fa0f8005c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.687 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:39:57.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8006260 0x7fa0f80086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f8009160 0x7fa0f8009530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:57.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa11eb4d700 1 -- 172.21.15.2:0/863644071 reap_dead start 2022-12-01T09:39:57.688 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8005c10 0x7fa0f80086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:57.689 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8009b10 0x7fa0f800bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:57.689 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1180c9650 con 0x7fa0f8009b10 2022-12-01T09:39:57.689 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f8008db0 con 0x7fa0f8009160 2022-12-01T09:39:57.689 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa117fff700 1 -- 172.21.15.2:0/863644071 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f800c6d0 con 0x7fa0f8005c10 2022-12-01T09:39:57.690 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.681+0000 7fa11eb4d700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.2:3300/0 conn(0x7fa0f8009b10 0x7fa0f800bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:57.690 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11db4b700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8005c10 0x7fa0f80086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:57.690 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11e34c700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f8009160 0x7fa0f8009530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:57.690 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11eb4d700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8005c10 msgr2=0x7fa0f80086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:39:57.690 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11eb4d700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8005c10 0x7fa0f80086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.691 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11eb4d700 1 -- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f8009160 msgr2=0x7fa0f8009530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:39:57.691 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11eb4d700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f8009160 0x7fa0f8009530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.691 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11eb4d700 1 -- 172.21.15.2:0/863644071 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180ce180 con 0x7fa0f8009b10 2022-12-01T09:39:57.692 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11eb4d700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8009b10 0x7fa0f800bf30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa1180cf420 tx=0x7fa108001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbedbd157ab350e server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:57.692 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11d34a700 1 -- 172.21.15.2:0/863644071 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108007520 con 0x7fa0f8009b10 2022-12-01T09:39:57.692 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11d34a700 1 -- 172.21.15.2:0/863644071 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa108007680 con 0x7fa0f8009b10 2022-12-01T09:39:57.692 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.682+0000 7fa11d34a700 1 -- 172.21.15.2:0/863644071 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108005a90 con 0x7fa0f8009b10 2022-12-01T09:39:57.692 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 -- 172.21.15.2:0/863644071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8009b10 msgr2=0x7fa0f800bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:57.693 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8009b10 0x7fa0f800bf30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa1180cf420 tx=0x7fa108001770 comp rx=0 tx=0).stop 2022-12-01T09:39:57.693 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 -- 172.21.15.2:0/863644071 shutdown_connections 2022-12-01T09:39:57.693 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8005c10 0x7fa0f80086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.693 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 --2- 172.21.15.2:0/863644071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8009b10 0x7fa0f800bf30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.694 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 --2- 172.21.15.2:0/863644071 >> v2:172.21.15.85:3300/0 conn(0x7fa0f8009160 0x7fa0f8009530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.694 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 -- 172.21.15.2:0/863644071 >> 172.21.15.2:0/863644071 conn(0x7fa1180ce8f0 msgr2=0x7fa1180cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:57.694 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 -- 172.21.15.2:0/863644071 shutdown_connections 2022-12-01T09:39:57.694 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.683+0000 7fa1205b0700 1 -- 172.21.15.2:0/863644071 wait complete. 2022-12-01T09:39:57.695 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.687+0000 7feafdd4a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-12-01T09:39:57.698 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.684+0000 7fa1205b0700 1 Processor -- start 2022-12-01T09:39:57.698 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.684+0000 7fa1205b0700 1 -- start start 2022-12-01T09:39:57.698 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa1205b0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8005c10 0x7fa1180b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:57.699 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa1205b0700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa0f8009160 0x7fa1180af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:57.699 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa1205b0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8009b10 0x7fa1181546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:57.699 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa1205b0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180c6be0 con 0x7fa0f8005c10 2022-12-01T09:39:57.699 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa1205b0700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180b4690 con 0x7fa0f8009160 2022-12-01T09:39:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa1205b0700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1180c6da0 con 0x7fa0f8009b10 2022-12-01T09:39:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa11e34c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8005c10 0x7fa1180b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa11e34c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8005c10 0x7fa1180b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60874/0 (socket says 172.21.15.2:60874) 2022-12-01T09:39:57.700 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa11e34c700 1 -- 172.21.15.2:0/1607466364 learned_addr learned my addr 172.21.15.2:0/1607466364 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:39:57.701 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa11eb4d700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8009b10 0x7fa1181546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:57.701 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa11db4b700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa0f8009160 0x7fa1180af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:57.701 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa11e34c700 1 -- 172.21.15.2:0/1607466364 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8009b10 msgr2=0x7fa1181546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:57.701 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.685+0000 7fa11e34c700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8009b10 0x7fa1181546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.702 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa11e34c700 1 -- 172.21.15.2:0/1607466364 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa0f8009160 msgr2=0x7fa1180af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:57.702 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa11e34c700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa0f8009160 0x7fa1180af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.702 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa11e34c700 1 -- 172.21.15.2:0/1607466364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11806bf80 con 0x7fa0f8005c10 2022-12-01T09:39:57.702 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa11e34c700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8005c10 0x7fa1180b4150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa110004640 tx=0x7fa110004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:57.703 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa116ffd700 1 -- 172.21.15.2:0/1607466364 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11001e070 con 0x7fa0f8005c10 2022-12-01T09:39:57.703 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa116ffd700 1 -- 172.21.15.2:0/1607466364 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11000f040 con 0x7fa0f8005c10 2022-12-01T09:39:57.703 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1181f5480 con 0x7fa0f8005c10 2022-12-01T09:39:57.703 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa116ffd700 1 -- 172.21.15.2:0/1607466364 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110019a50 con 0x7fa0f8005c10 2022-12-01T09:39:57.703 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1181f5a40 con 0x7fa0f8005c10 2022-12-01T09:39:57.704 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa116ffd700 1 -- 172.21.15.2:0/1607466364 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa110029020 con 0x7fa0f8005c10 2022-12-01T09:39:57.704 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.686+0000 7fa116ffd700 1 -- 172.21.15.2:0/1607466364 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa110018430 con 0x7fa0f8005c10 2022-12-01T09:39:57.704 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.687+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa118068b70 con 0x7fa0f8005c10 2022-12-01T09:39:57.704 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.689+0000 7fa116ffd700 1 -- 172.21.15.2:0/1607466364 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fa110019df0 con 0x7fa0f8005c10 2022-12-01T09:39:57.728 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:57.725+0000 7f73f316d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-12-01T09:39:57.764 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.761+0000 7feafdd4a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-12-01T09:39:57.768 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.765+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa1180ce460 con 0x7fa0f8005c10 2022-12-01T09:39:57.771 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.768+0000 7fa116ffd700 1 -- 172.21.15.2:0/1607466364 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa11002c430 con 0x7fa0f8005c10 2022-12-01T09:39:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.768+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8005c10 msgr2=0x7fa1180b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.769+0000 7fa1205b0700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8005c10 0x7fa1180b4150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa110004640 tx=0x7fa110004670 comp rx=0 tx=0).stop 2022-12-01T09:39:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.769+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 shutdown_connections 2022-12-01T09:39:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.769+0000 7fa1205b0700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fa0f8009b10 0x7fa1181546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.769+0000 7fa1205b0700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0f8005c10 0x7fa1180b4150 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.769+0000 7fa1205b0700 1 --2- 172.21.15.2:0/1607466364 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fa0f8009160 0x7fa1180af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.769+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 >> 172.21.15.2:0/1607466364 conn(0x7fa1180ce8f0 msgr2=0x7fa11815fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.769+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 shutdown_connections 2022-12-01T09:39:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:39:57.769+0000 7fa1205b0700 1 -- 172.21.15.2:0/1607466364 wait complete. 2022-12-01T09:39:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2022-12-01T09:39:57.784 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-12-01T09:39:57.785 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:57.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-12-01T09:39:57.808 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:57.805+0000 7f73f316d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-12-01T09:39:57.813 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:57.813 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-12-01T09:39:57.881 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:57.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-12-01T09:39:57.892 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.889+0000 7feafdd4a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-12-01T09:39:57.943 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:57.941+0000 7f73f316d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-12-01T09:39:57.948 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2022-12-01T09:39:57.948 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-12-01T09:39:57.979 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:57.976+0000 7feafdd4a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-12-01T09:39:58.015 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:58.016 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-12-01T09:39:58.029 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:58.026+0000 7f73f316d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-12-01T09:39:58.042 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:58.043 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-12-01T09:39:58.108 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:58.109 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-12-01T09:39:58.175 DEBUG:teuthology.orchestra.run.smithi085:> set -ex 2022-12-01T09:39:58.175 DEBUG:teuthology.orchestra.run.smithi085:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-12-01T09:39:58.241 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 29ad9800-3b0e-4728-9c4b-27eed3978c17 0 2022-12-01T09:39:58.401 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.398+0000 7fe827c49700 1 Processor -- start 2022-12-01T09:39:58.402 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.398+0000 7fe827c49700 1 -- start start 2022-12-01T09:39:58.404 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe827c49700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.404 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe827c49700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8200ce740 con 0x7fe82006fae0 2022-12-01T09:39:58.405 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe827c49700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82006bf80 con 0x7fe8200ca0c0 2022-12-01T09:39:58.405 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe827c49700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200cdf10 con 0x7fe8200cdaa0 2022-12-01T09:39:58.405 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe8251e4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.405 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe8259e5700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7fe8200ca0c0 0x7fe82006d3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:48304/0 (socket says 172.21.15.85:48304) 2022-12-01T09:39:58.406 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe8251e4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:35382/0 (socket says 172.21.15.85:35382) 2022-12-01T09:39:58.406 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe8259e5700 1 -- 172.21.15.85:0/1867964457 learned_addr learned my addr 172.21.15.85:0/1867964457 (peer_addr_for_me v1:172.21.15.85:0/0) 2022-12-01T09:39:58.406 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe8249e3700 1 -- 172.21.15.85:0/1867964457 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437074392 0 0) 0x7fe82006bf80 con 0x7fe8200ca0c0 2022-12-01T09:39:58.406 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe8251e4700 1 -- 172.21.15.85:0/1867964457 >> v1:172.21.15.85:6789/0 conn(0x7fe8200ca0c0 legacy=0x7fe82006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.407 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe8251e4700 1 -- 172.21.15.85:0/1867964457 >> v1:172.21.15.2:6789/0 conn(0x7fe82006fae0 legacy=0x7fe82006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.407 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.399+0000 7fe8251e4700 1 -- 172.21.15.85:0/1867964457 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200ce180 con 0x7fe8200cdaa0 2022-12-01T09:39:58.407 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe8251e4700 1 --2- 172.21.15.85:0/1867964457 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200c8f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe810009870 tx=0x7fe810009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93d9c464ac7495c0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:58.407 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe8249e3700 1 -- 172.21.15.85:0/1867964457 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810011830 con 0x7fe8200cdaa0 2022-12-01T09:39:58.408 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe8249e3700 1 -- 172.21.15.85:0/1867964457 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810011990 con 0x7fe8200cdaa0 2022-12-01T09:39:58.408 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe8249e3700 1 -- 172.21.15.85:0/1867964457 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810011d20 con 0x7fe8200cdaa0 2022-12-01T09:39:58.408 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe827c49700 1 -- 172.21.15.85:0/1867964457 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 msgr2=0x7fe8200c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.408 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe827c49700 1 --2- 172.21.15.85:0/1867964457 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200c8f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe810009870 tx=0x7fe810009b80 comp rx=0 tx=0).stop 2022-12-01T09:39:58.408 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe827c49700 1 -- 172.21.15.85:0/1867964457 shutdown_connections 2022-12-01T09:39:58.409 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe827c49700 1 --2- 172.21.15.85:0/1867964457 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200c8f40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.409 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.400+0000 7fe827c49700 1 -- 172.21.15.85:0/1867964457 >> 172.21.15.85:0/1867964457 conn(0x7fe8200ce8f0 msgr2=0x7fe8200cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:58.410 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.401+0000 7fe827c49700 1 -- 172.21.15.85:0/1867964457 shutdown_connections 2022-12-01T09:39:58.410 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.401+0000 7fe827c49700 1 -- 172.21.15.85:0/1867964457 wait complete. 2022-12-01T09:39:58.413 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.402+0000 7fe827c49700 1 Processor -- start 2022-12-01T09:39:58.414 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.402+0000 7fe827c49700 1 -- start start 2022-12-01T09:39:58.414 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.402+0000 7fe827c49700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe82006fae0 0x7fe820145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.414 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.402+0000 7fe827c49700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe8200ca0c0 0x7fe8200b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.414 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.402+0000 7fe827c49700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.415 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.402+0000 7fe827c49700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200c32c0 con 0x7fe82006fae0 2022-12-01T09:39:58.415 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.402+0000 7fe827c49700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200c29b0 con 0x7fe8200ca0c0 2022-12-01T09:39:58.415 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.402+0000 7fe827c49700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe820146240 con 0x7fe8200cdaa0 2022-12-01T09:39:58.415 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8251e4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe8200ca0c0 0x7fe8200b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.416 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8261e6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.416 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8251e4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe8200ca0c0 0x7fe8200b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42194/0 (socket says 172.21.15.85:42194) 2022-12-01T09:39:58.416 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8261e6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:35390/0 (socket says 172.21.15.85:35390) 2022-12-01T09:39:58.416 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8251e4700 1 -- 172.21.15.85:0/243823843 learned_addr learned my addr 172.21.15.85:0/243823843 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:58.416 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8259e5700 1 --2- 172.21.15.85:0/243823843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe82006fae0 0x7fe820145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.417 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8261e6700 1 -- 172.21.15.85:0/243823843 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe8200ca0c0 msgr2=0x7fe8200b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.417 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8261e6700 1 --2- 172.21.15.85:0/243823843 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe8200ca0c0 0x7fe8200b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.417 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8261e6700 1 -- 172.21.15.85:0/243823843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe82006fae0 msgr2=0x7fe820145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.417 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8261e6700 1 --2- 172.21.15.85:0/243823843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe82006fae0 0x7fe820145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.418 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8261e6700 1 -- 172.21.15.85:0/243823843 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100094a0 con 0x7fe8200cdaa0 2022-12-01T09:39:58.418 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe8261e6700 1 --2- 172.21.15.85:0/243823843 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200b1c60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe81800cb10 tx=0x7fe81800ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:58.418 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe80e7fc700 1 -- 172.21.15.85:0/243823843 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818003c00 con 0x7fe8200cdaa0 2022-12-01T09:39:58.419 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe80e7fc700 1 -- 172.21.15.85:0/243823843 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818003d60 con 0x7fe8200cdaa0 2022-12-01T09:39:58.420 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8200b44f0 con 0x7fe8200cdaa0 2022-12-01T09:39:58.420 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe80e7fc700 1 -- 172.21.15.85:0/243823843 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81800a9e0 con 0x7fe8200cdaa0 2022-12-01T09:39:58.420 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8200c2ee0 con 0x7fe8200cdaa0 2022-12-01T09:39:58.421 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe80e7fc700 1 -- 172.21.15.85:0/243823843 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe818010050 con 0x7fe8200cdaa0 2022-12-01T09:39:58.421 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.403+0000 7fe80e7fc700 1 -- 172.21.15.85:0/243823843 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe818012760 con 0x7fe8200cdaa0 2022-12-01T09:39:58.422 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.404+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8200c3400 con 0x7fe8200cdaa0 2022-12-01T09:39:58.422 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.406+0000 7fe80e7fc700 1 -- 172.21.15.85:0/243823843 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fe8200c3400 con 0x7fe8200cdaa0 2022-12-01T09:39:58.424 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:58.417+0000 7feafdd4a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-12-01T09:39:58.463 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:58.460+0000 7f73f316d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-12-01T09:39:58.493 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.490+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "29ad9800-3b0e-4728-9c4b-27eed3978c17", "id": 0} v 0) v1 -- 0x7fe820068b70 con 0x7fe8200cdaa0 2022-12-01T09:39:58.496 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.493+0000 7fe80e7fc700 1 -- 172.21.15.85:0/243823843 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "29ad9800-3b0e-4728-9c4b-27eed3978c17", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe818005800 con 0x7fe8200cdaa0 2022-12-01T09:39:58.497 INFO:teuthology.orchestra.run.smithi085.stdout:0 2022-12-01T09:39:58.498 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 msgr2=0x7fe8200b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.498 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 --2- 172.21.15.85:0/243823843 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200b1c60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe81800cb10 tx=0x7fe81800ce20 comp rx=0 tx=0).stop 2022-12-01T09:39:58.499 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 shutdown_connections 2022-12-01T09:39:58.499 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 --2- 172.21.15.85:0/243823843 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fe8200cdaa0 0x7fe8200b1c60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.499 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 --2- 172.21.15.85:0/243823843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe82006fae0 0x7fe820145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.499 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 --2- 172.21.15.85:0/243823843 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fe8200ca0c0 0x7fe8200b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.500 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 >> 172.21.15.85:0/243823843 conn(0x7fe8200ce8f0 msgr2=0x7fe82006d360 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:58.500 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 shutdown_connections 2022-12-01T09:39:58.500 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.494+0000 7fe827c49700 1 -- 172.21.15.85:0/243823843 wait complete. 2022-12-01T09:39:58.510 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 51463426-88ae-4c28-880b-1b655faff15c 1 2022-12-01T09:39:58.631 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.627+0000 7f5202865700 1 Processor -- start 2022-12-01T09:39:58.631 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.628+0000 7f5202865700 1 -- start start 2022-12-01T09:39:58.635 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.628+0000 7f5202865700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f51fc069570 0x7f51fc06c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.635 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.628+0000 7f5202865700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f51fc06c770 0x7f51fc15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.635 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.628+0000 7f5202865700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51fc0cde20 con 0x7f51fc06dad0 2022-12-01T09:39:58.636 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.628+0000 7f5202865700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc06cf80 con 0x7f51fc069570 2022-12-01T09:39:58.636 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.628+0000 7f5202865700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc15c3c0 con 0x7f51fc06c770 2022-12-01T09:39:58.636 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.628+0000 7f51fbfff700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f51fc06dad0 0x7f51fc068e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:48316/0 (socket says 172.21.15.85:48316) 2022-12-01T09:39:58.637 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.628+0000 7f51fbfff700 1 -- 172.21.15.85:0/2320276526 learned_addr learned my addr 172.21.15.85:0/2320276526 (peer_addr_for_me v1:172.21.15.85:0/0) 2022-12-01T09:39:58.637 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f51fb7fe700 1 --2- 172.21.15.85:0/2320276526 >> v2:172.21.15.2:3300/0 conn(0x7f51fc069570 0x7f51fc06c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.637 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f5200e02700 1 --2- 172.21.15.85:0/2320276526 >> v2:172.21.15.85:3300/0 conn(0x7f51fc06c770 0x7f51fc15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.638 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f51faffd700 1 -- 172.21.15.85:0/2320276526 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622092071 0 0) 0x7f51fc0cde20 con 0x7f51fc06dad0 2022-12-01T09:39:58.638 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f51faffd700 1 -- 172.21.15.85:0/2320276526 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e80035b0 con 0x7f51fc06dad0 2022-12-01T09:39:58.638 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f5200e02700 1 -- 172.21.15.85:0/2320276526 >> v2:172.21.15.2:3300/0 conn(0x7f51fc069570 msgr2=0x7f51fc06c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:39:58.638 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f5200e02700 1 --2- 172.21.15.85:0/2320276526 >> v2:172.21.15.2:3300/0 conn(0x7f51fc069570 0x7f51fc06c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.639 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f5200e02700 1 -- 172.21.15.85:0/2320276526 >> v1:172.21.15.85:6789/0 conn(0x7f51fc06dad0 legacy=0x7f51fc068e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.639 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f5200e02700 1 -- 172.21.15.85:0/2320276526 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc15c6a0 con 0x7f51fc06c770 2022-12-01T09:39:58.639 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f5200e02700 1 --2- 172.21.15.85:0/2320276526 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc06c770 0x7f51fc15f8a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0cc790 tx=0x7f51f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab9b2277fd7e1be7 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:58.640 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f51faffd700 1 -- 172.21.15.85:0/2320276526 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f400e070 con 0x7f51fc06c770 2022-12-01T09:39:58.640 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f51faffd700 1 -- 172.21.15.85:0/2320276526 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f400bb20 con 0x7f51fc06c770 2022-12-01T09:39:58.640 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.629+0000 7f51faffd700 1 -- 172.21.15.85:0/2320276526 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4012430 con 0x7f51fc06c770 2022-12-01T09:39:58.640 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.630+0000 7f5202865700 1 -- 172.21.15.85:0/2320276526 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc06c770 msgr2=0x7f51fc15f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.641 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.630+0000 7f5202865700 1 --2- 172.21.15.85:0/2320276526 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc06c770 0x7f51fc15f8a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0cc790 tx=0x7f51f400b330 comp rx=0 tx=0).stop 2022-12-01T09:39:58.641 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.630+0000 7f5202865700 1 -- 172.21.15.85:0/2320276526 shutdown_connections 2022-12-01T09:39:58.641 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.630+0000 7f5202865700 1 --2- 172.21.15.85:0/2320276526 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc06c770 0x7f51fc15f8a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.641 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.630+0000 7f5202865700 1 --2- 172.21.15.85:0/2320276526 >> v2:172.21.15.2:3300/0 conn(0x7f51fc069570 0x7f51fc06c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.642 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.630+0000 7f5202865700 1 -- 172.21.15.85:0/2320276526 >> 172.21.15.85:0/2320276526 conn(0x7f51fc0cbc00 msgr2=0x7f51fc0cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:58.642 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.630+0000 7f5202865700 1 -- 172.21.15.85:0/2320276526 shutdown_connections 2022-12-01T09:39:58.642 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.630+0000 7f5202865700 1 -- 172.21.15.85:0/2320276526 wait complete. 2022-12-01T09:39:58.642 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.632+0000 7f5202865700 1 Processor -- start 2022-12-01T09:39:58.643 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.632+0000 7f5202865700 1 -- start start 2022-12-01T09:39:58.646 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f5202865700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc069570 0x7f51fc1f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.646 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f5202865700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f51fc06c770 0x7f51fc0b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.646 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f5202865700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51fc06dad0 0x7f51fc0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.647 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f5202865700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0aecd0 con 0x7f51fc06dad0 2022-12-01T09:39:58.647 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f5202865700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0b45b0 con 0x7f51fc069570 2022-12-01T09:39:58.647 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f5202865700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0b4890 con 0x7f51fc06c770 2022-12-01T09:39:58.648 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f51fbfff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc069570 0x7f51fc1f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.648 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f51fbfff700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc069570 0x7f51fc1f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42204/0 (socket says 172.21.15.85:42204) 2022-12-01T09:39:58.648 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f51fbfff700 1 -- 172.21.15.85:0/1451103150 learned_addr learned my addr 172.21.15.85:0/1451103150 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:58.648 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f51fb7fe700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f51fc06c770 0x7f51fc0b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.649 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.633+0000 7f5200e02700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51fc06dad0 0x7f51fc0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.649 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51fbfff700 1 -- 172.21.15.85:0/1451103150 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f51fc06c770 msgr2=0x7f51fc0b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.649 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51fbfff700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f51fc06c770 0x7f51fc0b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.650 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51fbfff700 1 -- 172.21.15.85:0/1451103150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51fc06dad0 msgr2=0x7f51fc0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.650 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51fbfff700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51fc06dad0 0x7f51fc0b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.650 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51fbfff700 1 -- 172.21.15.85:0/1451103150 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4009a70 con 0x7f51fc069570 2022-12-01T09:39:58.650 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51fbfff700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc069570 0x7f51fc1f0d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51ec001eb0 tx=0x7f51ec00c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:58.651 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51f8ff9700 1 -- 172.21.15.85:0/1451103150 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec005d70 con 0x7f51fc069570 2022-12-01T09:39:58.651 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51f8ff9700 1 -- 172.21.15.85:0/1451103150 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec0070d0 con 0x7f51fc069570 2022-12-01T09:39:58.651 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51f8ff9700 1 -- 172.21.15.85:0/1451103150 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec007460 con 0x7f51fc069570 2022-12-01T09:39:58.652 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc0b4b70 con 0x7f51fc069570 2022-12-01T09:39:58.652 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc0b1880 con 0x7f51fc069570 2022-12-01T09:39:58.652 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51f8ff9700 1 -- 172.21.15.85:0/1451103150 <== mon.1 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f51ec019030 con 0x7f51fc069570 2022-12-01T09:39:58.652 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.634+0000 7f51f8ff9700 1 -- 172.21.15.85:0/1451103150 <== mon.1 v2:172.21.15.85:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f51ec01cd80 con 0x7f51fc069570 2022-12-01T09:39:58.653 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.635+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc0cd160 con 0x7f51fc069570 2022-12-01T09:39:58.653 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.639+0000 7f51f8ff9700 1 -- 172.21.15.85:0/1451103150 <== mon.1 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f51ec017070 con 0x7f51fc069570 2022-12-01T09:39:58.726 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.723+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "51463426-88ae-4c28-880b-1b655faff15c", "id": 1} v 0) v1 -- 0x7f51fc068b70 con 0x7f51fc069570 2022-12-01T09:39:58.729 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.726+0000 7f51f8ff9700 1 -- 172.21.15.85:0/1451103150 <== mon.1 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "51463426-88ae-4c28-880b-1b655faff15c", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f51ec01c510 con 0x7f51fc069570 2022-12-01T09:39:58.730 INFO:teuthology.orchestra.run.smithi085.stdout:1 2022-12-01T09:39:58.731 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc069570 msgr2=0x7f51fc1f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.732 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc069570 0x7f51fc1f0d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51ec001eb0 tx=0x7f51ec00c7b0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.732 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 shutdown_connections 2022-12-01T09:39:58.732 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51fc06dad0 0x7f51fc0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.732 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f51fc069570 0x7f51fc1f0d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.733 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 --2- 172.21.15.85:0/1451103150 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f51fc06c770 0x7f51fc0b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.733 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 >> 172.21.15.85:0/1451103150 conn(0x7f51fc0cbc00 msgr2=0x7f51fc068e60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:58.733 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 shutdown_connections 2022-12-01T09:39:58.733 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.727+0000 7f5202865700 1 -- 172.21.15.85:0/1451103150 wait complete. 2022-12-01T09:39:58.742 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 817f70a8-c4b5-4470-beca-45061a2bf389 2 2022-12-01T09:39:58.899 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.887+0000 7f4e5d4d6700 1 Processor -- start 2022-12-01T09:39:58.900 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.887+0000 7f4e5d4d6700 1 -- start start 2022-12-01T09:39:58.900 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e5d4d6700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4e58152710 0x7f4e580cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.900 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e5d4d6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e5815c340 0x7f4e5815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.901 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e5d4d6700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580cce40 con 0x7f4e581586d0 2022-12-01T09:39:58.901 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e5d4d6700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5815be40 con 0x7f4e58152710 2022-12-01T09:39:58.902 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e5d4d6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58158eb0 con 0x7f4e5815c340 2022-12-01T09:39:58.902 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e567fc700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f4e581586d0 0x7f4e580c8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:48330/0 (socket says 172.21.15.85:48330) 2022-12-01T09:39:58.902 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e567fc700 1 -- 172.21.15.85:0/820870371 learned_addr learned my addr 172.21.15.85:0/820870371 (peer_addr_for_me v1:172.21.15.85:0/0) 2022-12-01T09:39:58.903 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e577fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e5815c340 0x7f4e5815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.903 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e56ffd700 1 --2- 172.21.15.85:0/820870371 >> v2:172.21.15.2:3300/0 conn(0x7f4e58152710 0x7f4e580cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.903 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e55ffb700 1 -- 172.21.15.85:0/820870371 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567075586 0 0) 0x7f4e580cce40 con 0x7f4e581586d0 2022-12-01T09:39:58.904 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e577fe700 1 -- 172.21.15.85:0/820870371 >> v2:172.21.15.2:3300/0 conn(0x7f4e58152710 msgr2=0x7f4e580cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:39:58.904 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e577fe700 1 --2- 172.21.15.85:0/820870371 >> v2:172.21.15.2:3300/0 conn(0x7f4e58152710 0x7f4e580cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.904 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e577fe700 1 -- 172.21.15.85:0/820870371 >> v1:172.21.15.85:6789/0 conn(0x7f4e581586d0 legacy=0x7f4e580c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.905 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e577fe700 1 -- 172.21.15.85:0/820870371 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58159190 con 0x7f4e5815c340 2022-12-01T09:39:58.905 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e577fe700 1 --2- 172.21.15.85:0/820870371 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e5815c340 0x7f4e5815c7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009d90 tx=0x7f4e5000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cc87cee4a14371 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:58.905 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e55ffb700 1 -- 172.21.15.85:0/820870371 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000e070 con 0x7f4e5815c340 2022-12-01T09:39:58.905 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e55ffb700 1 -- 172.21.15.85:0/820870371 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5000bbd0 con 0x7f4e5815c340 2022-12-01T09:39:58.906 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.888+0000 7f4e55ffb700 1 -- 172.21.15.85:0/820870371 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50012430 con 0x7f4e5815c340 2022-12-01T09:39:58.906 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.889+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/820870371 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e5815c340 msgr2=0x7f4e5815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.906 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.889+0000 7f4e5d4d6700 1 --2- 172.21.15.85:0/820870371 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e5815c340 0x7f4e5815c7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009d90 tx=0x7f4e5000b330 comp rx=0 tx=0).stop 2022-12-01T09:39:58.906 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.889+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/820870371 shutdown_connections 2022-12-01T09:39:58.907 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.889+0000 7f4e5d4d6700 1 --2- 172.21.15.85:0/820870371 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e5815c340 0x7f4e5815c7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.907 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.889+0000 7f4e5d4d6700 1 --2- 172.21.15.85:0/820870371 >> v2:172.21.15.2:3300/0 conn(0x7f4e58152710 0x7f4e580cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.907 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.889+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/820870371 >> 172.21.15.85:0/820870371 conn(0x7f4e580ce8f0 msgr2=0x7f4e580cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:58.907 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.889+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/820870371 shutdown_connections 2022-12-01T09:39:58.908 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.889+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/820870371 wait complete. 2022-12-01T09:39:58.908 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.890+0000 7f4e5d4d6700 1 Processor -- start 2022-12-01T09:39:58.908 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.890+0000 7f4e5d4d6700 1 -- start start 2022-12-01T09:39:58.908 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e5d4d6700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4e58152710 0x7f4e581f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.908 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e5d4d6700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e581586d0 0x7f4e580c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.909 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e5d4d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e5815c340 0x7f4e5815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:58.909 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e5d4d6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58159ee0 con 0x7f4e5815c340 2022-12-01T09:39:58.909 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e5d4d6700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5815a1c0 con 0x7f4e58152710 2022-12-01T09:39:58.909 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e5d4d6700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5815a4a0 con 0x7f4e581586d0 2022-12-01T09:39:58.910 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e567fc700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e581586d0 0x7f4e580c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.910 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e56ffd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4e58152710 0x7f4e581f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.910 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e567fc700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e581586d0 0x7f4e580c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:35412/0 (socket says 172.21.15.85:35412) 2022-12-01T09:39:58.910 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e56ffd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4e58152710 0x7f4e581f1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42216/0 (socket says 172.21.15.85:42216) 2022-12-01T09:39:58.910 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e567fc700 1 -- 172.21.15.85:0/2967360374 learned_addr learned my addr 172.21.15.85:0/2967360374 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:58.911 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.891+0000 7f4e577fe700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e5815c340 0x7f4e5815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:58.911 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.892+0000 7f4e567fc700 1 -- 172.21.15.85:0/2967360374 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4e58152710 msgr2=0x7f4e581f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.911 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.892+0000 7f4e567fc700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4e58152710 0x7f4e581f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.911 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.892+0000 7f4e567fc700 1 -- 172.21.15.85:0/2967360374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e5815c340 msgr2=0x7f4e5815af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.912 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.892+0000 7f4e567fc700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e5815c340 0x7f4e5815af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.912 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.892+0000 7f4e567fc700 1 -- 172.21.15.85:0/2967360374 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e50009a70 con 0x7f4e581586d0 2022-12-01T09:39:58.952 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.892+0000 7f4e56ffd700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4e58152710 0x7f4e581f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:39:58.953 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.892+0000 7f4e567fc700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e581586d0 0x7f4e580c59c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0011e0 tx=0x7f4e4c00ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:58.953 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.893+0000 7f4e3f7fe700 1 -- 172.21.15.85:0/2967360374 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c004c80 con 0x7f4e581586d0 2022-12-01T09:39:58.953 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.893+0000 7f4e3f7fe700 1 -- 172.21.15.85:0/2967360374 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4c004e00 con 0x7f4e581586d0 2022-12-01T09:39:58.954 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.893+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5815a780 con 0x7f4e581586d0 2022-12-01T09:39:58.954 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.893+0000 7f4e3f7fe700 1 -- 172.21.15.85:0/2967360374 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c007310 con 0x7f4e581586d0 2022-12-01T09:39:58.954 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.893+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e581f1660 con 0x7f4e581586d0 2022-12-01T09:39:58.954 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.893+0000 7f4e3f7fe700 1 -- 172.21.15.85:0/2967360374 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4e4c01a020 con 0x7f4e581586d0 2022-12-01T09:39:58.955 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.893+0000 7f4e3f7fe700 1 -- 172.21.15.85:0/2967360374 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f4e4c0137e0 con 0x7f4e581586d0 2022-12-01T09:39:58.955 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.894+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e58068b70 con 0x7f4e581586d0 2022-12-01T09:39:58.955 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.896+0000 7f4e3f7fe700 1 -- 172.21.15.85:0/2967360374 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f4e4c007570 con 0x7f4e581586d0 2022-12-01T09:39:58.980 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.977+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "817f70a8-c4b5-4470-beca-45061a2bf389", "id": 2} v 0) v1 -- 0x7f4e580b4870 con 0x7f4e581586d0 2022-12-01T09:39:58.986 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.983+0000 7f4e3f7fe700 1 -- 172.21.15.85:0/2967360374 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "817f70a8-c4b5-4470-beca-45061a2bf389", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e4c01d4c0 con 0x7f4e581586d0 2022-12-01T09:39:58.986 INFO:teuthology.orchestra.run.smithi085.stdout:2 2022-12-01T09:39:58.988 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e581586d0 msgr2=0x7f4e580c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:58.988 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e581586d0 0x7f4e580c59c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0011e0 tx=0x7f4e4c00ca30 comp rx=0 tx=0).stop 2022-12-01T09:39:58.988 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 shutdown_connections 2022-12-01T09:39:58.988 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e5815c340 0x7f4e5815af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.989 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4e58152710 0x7f4e581f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.989 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 --2- 172.21.15.85:0/2967360374 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4e581586d0 0x7f4e580c59c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:58.989 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 >> 172.21.15.85:0/2967360374 conn(0x7f4e580ce8f0 msgr2=0x7f4e580cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:58.989 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 shutdown_connections 2022-12-01T09:39:58.990 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:58.984+0000 7f4e5d4d6700 1 -- 172.21.15.85:0/2967360374 wait complete. 2022-12-01T09:39:58.995 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:58.992+0000 7feafdd4a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-12-01T09:39:58.999 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 379bf437-773e-4be5-ac5c-1a8c6b6663e1 3 2022-12-01T09:39:59.048 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:59.045+0000 7f73f316d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-12-01T09:39:59.123 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.120+0000 7fb295afe700 1 Processor -- start 2022-12-01T09:39:59.124 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.120+0000 7fb295afe700 1 -- start start 2022-12-01T09:39:59.129 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.120+0000 7fb295afe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015bac0 0x7fb29006a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.130 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.120+0000 7fb295afe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb29015be90 0x7fb2900cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.130 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.120+0000 7fb295afe700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb2900ccde0 0x7fb29015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.130 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.120+0000 7fb295afe700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2900cd260 con 0x7fb29015be90 2022-12-01T09:39:59.130 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.120+0000 7fb295afe700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2900cf530 con 0x7fb2900ccde0 2022-12-01T09:39:59.131 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.120+0000 7fb295afe700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29006b910 con 0x7fb29015bac0 2022-12-01T09:39:59.131 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015bac0 0x7fb29006a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.131 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28ffff700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb2900ccde0 0x7fb29015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.131 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015bac0 0x7fb29006a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:35416/0 (socket says 172.21.15.85:35416) 2022-12-01T09:39:59.132 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 -- 172.21.15.85:0/696616340 learned_addr learned my addr 172.21.15.85:0/696616340 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:59.132 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28ffff700 1 --2- 172.21.15.85:0/696616340 >> v2:172.21.15.85:3300/0 conn(0x7fb2900ccde0 0x7fb29015f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42226/0 (socket says 172.21.15.85:42226) 2022-12-01T09:39:59.132 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28effd700 1 --2- 172.21.15.85:0/696616340 >> v2:172.21.15.2:3300/0 conn(0x7fb29015be90 0x7fb2900cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.132 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 -- 172.21.15.85:0/696616340 >> v2:172.21.15.85:3300/0 conn(0x7fb2900ccde0 msgr2=0x7fb29015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:39:59.133 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 --2- 172.21.15.85:0/696616340 >> v2:172.21.15.85:3300/0 conn(0x7fb2900ccde0 0x7fb29015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.133 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 -- 172.21.15.85:0/696616340 >> v2:172.21.15.2:3300/0 conn(0x7fb29015be90 msgr2=0x7fb2900cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:39:59.133 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 --2- 172.21.15.85:0/696616340 >> v2:172.21.15.2:3300/0 conn(0x7fb29015be90 0x7fb2900cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.133 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 -- 172.21.15.85:0/696616340 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29006bbf0 con 0x7fb29015bac0 2022-12-01T09:39:59.134 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28ffff700 1 --2- 172.21.15.85:0/696616340 >> v2:172.21.15.85:3300/0 conn(0x7fb2900ccde0 0x7fb29015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:39:59.134 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28f7fe700 1 --2- 172.21.15.85:0/696616340 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015bac0 0x7fb29006a980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb29006e700 tx=0x7fb280009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e25d845149c752b server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:59.134 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28e7fc700 1 -- 172.21.15.85:0/696616340 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280011830 con 0x7fb29015bac0 2022-12-01T09:39:59.134 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28e7fc700 1 -- 172.21.15.85:0/696616340 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb280011990 con 0x7fb29015bac0 2022-12-01T09:39:59.134 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.121+0000 7fb28e7fc700 1 -- 172.21.15.85:0/696616340 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280011d20 con 0x7fb29015bac0 2022-12-01T09:39:59.135 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 -- 172.21.15.85:0/696616340 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015bac0 msgr2=0x7fb29006a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.135 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 --2- 172.21.15.85:0/696616340 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015bac0 0x7fb29006a980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb29006e700 tx=0x7fb280009bd0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.135 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 -- 172.21.15.85:0/696616340 shutdown_connections 2022-12-01T09:39:59.135 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 --2- 172.21.15.85:0/696616340 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015bac0 0x7fb29006a980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.136 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 --2- 172.21.15.85:0/696616340 >> v2:172.21.15.85:3300/0 conn(0x7fb2900ccde0 0x7fb29015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.136 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 --2- 172.21.15.85:0/696616340 >> v2:172.21.15.2:3300/0 conn(0x7fb29015be90 0x7fb2900cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.136 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 -- 172.21.15.85:0/696616340 >> 172.21.15.85:0/696616340 conn(0x7fb290064ec0 msgr2=0x7fb290065290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:59.136 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 -- 172.21.15.85:0/696616340 shutdown_connections 2022-12-01T09:39:59.136 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.122+0000 7fb295afe700 1 -- 172.21.15.85:0/696616340 wait complete. 2022-12-01T09:39:59.137 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.123+0000 7fb295afe700 1 Processor -- start 2022-12-01T09:39:59.137 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.123+0000 7fb295afe700 1 -- start start 2022-12-01T09:39:59.137 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb295afe700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2900ccde0 0x7fb2901f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.138 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb295afe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb29015bac0 0x7fb290153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.138 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb295afe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015be90 0x7fb2901e3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.138 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb295afe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901e45f0 con 0x7fb29015bac0 2022-12-01T09:39:59.138 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb295afe700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290153d80 con 0x7fb2900ccde0 2022-12-01T09:39:59.138 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb295afe700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901e3970 con 0x7fb29015be90 2022-12-01T09:39:59.139 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28ffff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015be90 0x7fb2901e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.139 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2900ccde0 0x7fb2901f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.139 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28ffff700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015be90 0x7fb2901e3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:35426/0 (socket says 172.21.15.85:35426) 2022-12-01T09:39:59.139 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28ffff700 1 -- 172.21.15.85:0/1070290562 learned_addr learned my addr 172.21.15.85:0/1070290562 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:59.140 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28effd700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb29015bac0 0x7fb290153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.140 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28ffff700 1 -- 172.21.15.85:0/1070290562 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2900ccde0 msgr2=0x7fb2901f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.140 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28ffff700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2900ccde0 0x7fb2901f1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.140 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28ffff700 1 -- 172.21.15.85:0/1070290562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb29015bac0 msgr2=0x7fb290153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.141 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28ffff700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb29015bac0 0x7fb290153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.141 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28ffff700 1 -- 172.21.15.85:0/1070290562 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288009a70 con 0x7fb29015be90 2022-12-01T09:39:59.141 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.124+0000 7fb28f7fe700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2900ccde0 0x7fb2901f1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:39:59.141 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.125+0000 7fb28ffff700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015be90 0x7fb2901e3100 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb28800eed0 tx=0x7fb28800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:59.142 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.125+0000 7fb277fff700 1 -- 172.21.15.85:0/1070290562 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288010040 con 0x7fb29015be90 2022-12-01T09:39:59.142 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.125+0000 7fb277fff700 1 -- 172.21.15.85:0/1070290562 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28800ba80 con 0x7fb29015be90 2022-12-01T09:39:59.142 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.125+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb280009550 con 0x7fb29015be90 2022-12-01T09:39:59.142 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.125+0000 7fb277fff700 1 -- 172.21.15.85:0/1070290562 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800be10 con 0x7fb29015be90 2022-12-01T09:39:59.142 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.125+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2901e3f30 con 0x7fb29015be90 2022-12-01T09:39:59.143 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.125+0000 7fb277fff700 1 -- 172.21.15.85:0/1070290562 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fb288003a20 con 0x7fb29015be90 2022-12-01T09:39:59.143 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.125+0000 7fb277fff700 1 -- 172.21.15.85:0/1070290562 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fb2880046b0 con 0x7fb29015be90 2022-12-01T09:39:59.143 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.126+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb290068b70 con 0x7fb29015be90 2022-12-01T09:39:59.144 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.128+0000 7fb277fff700 1 -- 172.21.15.85:0/1070290562 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fb288018030 con 0x7fb29015be90 2022-12-01T09:39:59.157 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:59.154+0000 7feafdd4a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-12-01T09:39:59.208 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:59.205+0000 7f73f316d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-12-01T09:39:59.214 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.211+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "379bf437-773e-4be5-ac5c-1a8c6b6663e1", "id": 3} v 0) v1 -- 0x7fb2900cf7a0 con 0x7fb29015be90 2022-12-01T09:39:59.223 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.219+0000 7fb277fff700 1 -- 172.21.15.85:0/1070290562 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "379bf437-773e-4be5-ac5c-1a8c6b6663e1", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb288004020 con 0x7fb29015be90 2022-12-01T09:39:59.223 INFO:teuthology.orchestra.run.smithi085.stdout:3 2022-12-01T09:39:59.224 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015be90 msgr2=0x7fb2901e3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.225 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015be90 0x7fb2901e3100 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb28800eed0 tx=0x7fb28800b390 comp rx=0 tx=0).stop 2022-12-01T09:39:59.225 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 shutdown_connections 2022-12-01T09:39:59.225 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb29015be90 0x7fb2901e3100 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.225 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb2900ccde0 0x7fb2901f1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.226 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 --2- 172.21.15.85:0/1070290562 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb29015bac0 0x7fb290153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.226 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 >> 172.21.15.85:0/1070290562 conn(0x7fb290064ec0 msgr2=0x7fb2900b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:59.226 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 shutdown_connections 2022-12-01T09:39:59.226 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.220+0000 7fb295afe700 1 -- 172.21.15.85:0/1070290562 wait complete. 2022-12-01T09:39:59.235 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 7ef04bad-dece-4880-9f9c-4ba20e210e0e 4 2022-12-01T09:39:59.355 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.352+0000 7fed15432700 1 Processor -- start 2022-12-01T09:39:59.356 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.352+0000 7fed15432700 1 -- start start 2022-12-01T09:39:59.363 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.352+0000 7fed15432700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fed100b12c0 0x7fed100cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.363 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.352+0000 7fed15432700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed100c9880 con 0x7fed100cb760 2022-12-01T09:39:59.363 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.352+0000 7fed15432700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed10071110 con 0x7fed100cad00 2022-12-01T09:39:59.364 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.352+0000 7fed15432700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1015d690 con 0x7fed100b12c0 2022-12-01T09:39:59.364 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0f7fe700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7fed100cad00 0x7fed10070a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:48356/0 (socket says 172.21.15.85:48356) 2022-12-01T09:39:59.364 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0f7fe700 1 -- 172.21.15.85:0/145186430 learned_addr learned my addr 172.21.15.85:0/145186430 (peer_addr_for_me v1:172.21.15.85:0/0) 2022-12-01T09:39:59.364 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0effd700 1 --2- 172.21.15.85:0/145186430 >> v2:172.21.15.2:3300/0 conn(0x7fed100b12c0 0x7fed100cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.365 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0dffb700 1 -- 172.21.15.85:0/145186430 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732572398 0 0) 0x7fed10071110 con 0x7fed100cad00 2022-12-01T09:39:59.365 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0dffb700 1 -- 172.21.15.85:0/145186430 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecfc0035b0 con 0x7fed100cad00 2022-12-01T09:39:59.365 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0dffb700 1 -- 172.21.15.85:0/145186430 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306193758 0 0) 0x7fed100c9880 con 0x7fed100cb760 2022-12-01T09:39:59.365 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0dffb700 1 -- 172.21.15.85:0/145186430 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed10071110 con 0x7fed100cb760 2022-12-01T09:39:59.366 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0effd700 1 -- 172.21.15.85:0/145186430 >> v1:172.21.15.85:6789/0 conn(0x7fed100cad00 legacy=0x7fed10070a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.366 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0effd700 1 -- 172.21.15.85:0/145186430 >> v1:172.21.15.2:6789/0 conn(0x7fed100cb760 legacy=0x7fed100702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.366 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0effd700 1 -- 172.21.15.85:0/145186430 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1015cb10 con 0x7fed100b12c0 2022-12-01T09:39:59.366 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0effd700 1 --2- 172.21.15.85:0/145186430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 0x7fed100cd180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed00009840 tx=0x7fed00009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8ce729a505b01b server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:59.367 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0dffb700 1 -- 172.21.15.85:0/145186430 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00011830 con 0x7fed100b12c0 2022-12-01T09:39:59.367 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0dffb700 1 -- 172.21.15.85:0/145186430 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed00011990 con 0x7fed100b12c0 2022-12-01T09:39:59.367 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.353+0000 7fed0dffb700 1 -- 172.21.15.85:0/145186430 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00011d20 con 0x7fed100b12c0 2022-12-01T09:39:59.367 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.354+0000 7fed15432700 1 -- 172.21.15.85:0/145186430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 msgr2=0x7fed100cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.368 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.354+0000 7fed15432700 1 --2- 172.21.15.85:0/145186430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 0x7fed100cd180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fed00009840 tx=0x7fed00009b50 comp rx=0 tx=0).stop 2022-12-01T09:39:59.368 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.354+0000 7fed15432700 1 -- 172.21.15.85:0/145186430 shutdown_connections 2022-12-01T09:39:59.368 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.354+0000 7fed15432700 1 --2- 172.21.15.85:0/145186430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 0x7fed100cd180 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.368 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.354+0000 7fed15432700 1 -- 172.21.15.85:0/145186430 >> 172.21.15.85:0/145186430 conn(0x7fed100cbc00 msgr2=0x7fed100cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:59.369 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.354+0000 7fed15432700 1 -- 172.21.15.85:0/145186430 shutdown_connections 2022-12-01T09:39:59.369 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.354+0000 7fed15432700 1 -- 172.21.15.85:0/145186430 wait complete. 2022-12-01T09:39:59.369 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.355+0000 7fed15432700 1 Processor -- start 2022-12-01T09:39:59.369 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed15432700 1 -- start start 2022-12-01T09:39:59.370 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed15432700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 0x7fed100b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.370 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed15432700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fed100cad00 0x7fed10142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.370 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed15432700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fed100cb760 0x7fed10145250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.370 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed15432700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100b4f40 con 0x7fed100b12c0 2022-12-01T09:39:59.371 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed15432700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100b4730 con 0x7fed100cad00 2022-12-01T09:39:59.371 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed15432700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed100b4980 con 0x7fed100cb760 2022-12-01T09:39:59.371 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed0e7fc700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fed100cad00 0x7fed10142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.371 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed0e7fc700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fed100cad00 0x7fed10142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42242/0 (socket says 172.21.15.85:42242) 2022-12-01T09:39:59.372 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed0e7fc700 1 -- 172.21.15.85:0/3570070057 learned_addr learned my addr 172.21.15.85:0/3570070057 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:59.372 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed0effd700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 0x7fed100b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.372 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.356+0000 7fed0f7fe700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fed100cb760 0x7fed10145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.372 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed0e7fc700 1 -- 172.21.15.85:0/3570070057 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fed100cb760 msgr2=0x7fed10145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.373 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed0e7fc700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fed100cb760 0x7fed10145250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.373 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed0e7fc700 1 -- 172.21.15.85:0/3570070057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 msgr2=0x7fed100b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.373 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed0e7fc700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 0x7fed100b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.373 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed0e7fc700 1 -- 172.21.15.85:0/3570070057 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000094a0 con 0x7fed100cad00 2022-12-01T09:39:59.374 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed0effd700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 0x7fed100b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:39:59.374 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed0e7fc700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fed100cad00 0x7fed10142040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed0400c520 tx=0x7fed0400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:59.374 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fecf77fe700 1 -- 172.21.15.85:0/3570070057 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed040052d0 con 0x7fed100cad00 2022-12-01T09:39:59.374 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fecf77fe700 1 -- 172.21.15.85:0/3570070057 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04005430 con 0x7fed100cad00 2022-12-01T09:39:59.375 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed100b4c60 con 0x7fed100cad00 2022-12-01T09:39:59.375 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fecf77fe700 1 -- 172.21.15.85:0/3570070057 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed040038c0 con 0x7fed100cad00 2022-12-01T09:39:59.375 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed100b24f0 con 0x7fed100cad00 2022-12-01T09:39:59.375 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fecf77fe700 1 -- 172.21.15.85:0/3570070057 <== mon.1 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fed04019030 con 0x7fed100cad00 2022-12-01T09:39:59.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.357+0000 7fecf77fe700 1 -- 172.21.15.85:0/3570070057 <== mon.1 v2:172.21.15.85:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fed04011db0 con 0x7fed100cad00 2022-12-01T09:39:59.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.358+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed100abd90 con 0x7fed100cad00 2022-12-01T09:39:59.376 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.360+0000 7fecf77fe700 1 -- 172.21.15.85:0/3570070057 <== mon.1 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fed100abd90 con 0x7fed100cad00 2022-12-01T09:39:59.444 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.441+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7ef04bad-dece-4880-9f9c-4ba20e210e0e", "id": 4} v 0) v1 -- 0x7fed100ac170 con 0x7fed100cad00 2022-12-01T09:39:59.452 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.449+0000 7fecf77fe700 1 -- 172.21.15.85:0/3570070057 <== mon.1 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ef04bad-dece-4880-9f9c-4ba20e210e0e", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fed04017070 con 0x7fed100cad00 2022-12-01T09:39:59.453 INFO:teuthology.orchestra.run.smithi085.stdout:4 2022-12-01T09:39:59.454 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fed100cad00 msgr2=0x7fed10142040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.454 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fed100cad00 0x7fed10142040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed0400c520 tx=0x7fed0400c830 comp rx=0 tx=0).stop 2022-12-01T09:39:59.455 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 shutdown_connections 2022-12-01T09:39:59.455 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fed100cb760 0x7fed10145250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.455 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed100b12c0 0x7fed100b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.455 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 --2- 172.21.15.85:0/3570070057 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fed100cad00 0x7fed10142040 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.456 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 >> 172.21.15.85:0/3570070057 conn(0x7fed100cbc00 msgr2=0x7fed10158520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:59.456 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 shutdown_connections 2022-12-01T09:39:59.456 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.450+0000 7fed15432700 1 -- 172.21.15.85:0/3570070057 wait complete. 2022-12-01T09:39:59.465 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new 9f624f12-fff1-44e7-b4f2-76065630d976 5 2022-12-01T09:39:59.584 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.580+0000 7fd0ff279700 1 Processor -- start 2022-12-01T09:39:59.584 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.580+0000 7fd0ff279700 1 -- start start 2022-12-01T09:39:59.587 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0ff279700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd0f80b12c0 0x7fd0f80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.588 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0ff279700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd0f80cad00 0x7fd0f80cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.588 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0ff279700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f80c9880 con 0x7fd0f80cb760 2022-12-01T09:39:59.588 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0ff279700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f815d690 con 0x7fd0f80cad00 2022-12-01T09:39:59.588 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0ff279700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f815cb10 con 0x7fd0f80b12c0 2022-12-01T09:39:59.589 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0fd015700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd0f80b12c0 0x7fd0f80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.589 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0fd015700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fd0f80b12c0 0x7fd0f80cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42246/0 (socket says 172.21.15.85:42246) 2022-12-01T09:39:59.589 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0fd015700 1 -- 172.21.15.85:0/2574547976 learned_addr learned my addr 172.21.15.85:0/2574547976 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:59.590 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0fd816700 1 --2- 172.21.15.85:0/2574547976 >> v2:172.21.15.2:3300/0 conn(0x7fd0f80cad00 0x7fd0f80cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.590 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0fd015700 1 -- 172.21.15.85:0/2574547976 >> v2:172.21.15.2:3300/0 conn(0x7fd0f80cad00 msgr2=0x7fd0f80cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:39:59.590 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0fd015700 1 --2- 172.21.15.85:0/2574547976 >> v2:172.21.15.2:3300/0 conn(0x7fd0f80cad00 0x7fd0f80cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.590 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0fd015700 1 -- 172.21.15.85:0/2574547976 >> v1:172.21.15.2:6789/0 conn(0x7fd0f80cb760 legacy=0x7fd0f80702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.591 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.581+0000 7fd0fd015700 1 -- 172.21.15.85:0/2574547976 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f815cdf0 con 0x7fd0f80b12c0 2022-12-01T09:39:59.591 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0fd816700 1 --2- 172.21.15.85:0/2574547976 >> v2:172.21.15.2:3300/0 conn(0x7fd0f80cad00 0x7fd0f80cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:39:59.591 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0fd015700 1 --2- 172.21.15.85:0/2574547976 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80b12c0 0x7fd0f80cd180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd0f806faf0 tx=0x7fd0f400be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d9ea8fde0578584 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:59.591 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0e7fff700 1 -- 172.21.15.85:0/2574547976 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400dd30 con 0x7fd0f80b12c0 2022-12-01T09:39:59.592 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0e7fff700 1 -- 172.21.15.85:0/2574547976 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f400de90 con 0x7fd0f80b12c0 2022-12-01T09:39:59.592 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0e7fff700 1 -- 172.21.15.85:0/2574547976 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4010690 con 0x7fd0f80b12c0 2022-12-01T09:39:59.592 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0ff279700 1 -- 172.21.15.85:0/2574547976 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80b12c0 msgr2=0x7fd0f80cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.592 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0ff279700 1 --2- 172.21.15.85:0/2574547976 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80b12c0 0x7fd0f80cd180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd0f806faf0 tx=0x7fd0f400be70 comp rx=0 tx=0).stop 2022-12-01T09:39:59.593 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0ff279700 1 -- 172.21.15.85:0/2574547976 shutdown_connections 2022-12-01T09:39:59.593 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0ff279700 1 --2- 172.21.15.85:0/2574547976 >> v2:172.21.15.2:3300/0 conn(0x7fd0f80cad00 0x7fd0f80cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.593 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0ff279700 1 --2- 172.21.15.85:0/2574547976 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80b12c0 0x7fd0f80cd180 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.593 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.582+0000 7fd0ff279700 1 -- 172.21.15.85:0/2574547976 >> 172.21.15.85:0/2574547976 conn(0x7fd0f80cbc00 msgr2=0x7fd0f80cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:59.594 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.583+0000 7fd0ff279700 1 -- 172.21.15.85:0/2574547976 shutdown_connections 2022-12-01T09:39:59.594 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.583+0000 7fd0ff279700 1 -- 172.21.15.85:0/2574547976 wait complete. 2022-12-01T09:39:59.594 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.584+0000 7fd0ff279700 1 Processor -- start 2022-12-01T09:39:59.594 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.584+0000 7fd0ff279700 1 -- start start 2022-12-01T09:39:59.597 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0ff279700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0f80b12c0 0x7fd0f814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.598 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0ff279700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd0f80cad00 0x7fd0f814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.598 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0ff279700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80cb760 0x7fd0f814e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.598 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0ff279700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f814c750 con 0x7fd0f80b12c0 2022-12-01T09:39:59.599 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0ff279700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f814dc20 con 0x7fd0f80cb760 2022-12-01T09:39:59.599 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0ff279700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f814d660 con 0x7fd0f80cad00 2022-12-01T09:39:59.599 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd816700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80cb760 0x7fd0f814e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.599 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd015700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0f80b12c0 0x7fd0f814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.600 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd816700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80cb760 0x7fd0f814e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42254/0 (socket says 172.21.15.85:42254) 2022-12-01T09:39:59.600 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd015700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0f80b12c0 0x7fd0f814c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.85:60656/0 (socket says 172.21.15.85:60656) 2022-12-01T09:39:59.600 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd816700 1 -- 172.21.15.85:0/1337481630 learned_addr learned my addr 172.21.15.85:0/1337481630 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:59.601 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fc814700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd0f80cad00 0x7fd0f814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.601 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd816700 1 -- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd0f80cad00 msgr2=0x7fd0f814de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.602 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd816700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd0f80cad00 0x7fd0f814de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.602 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd816700 1 -- 172.21.15.85:0/1337481630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0f80b12c0 msgr2=0x7fd0f814c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.602 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.585+0000 7fd0fd816700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0f80b12c0 0x7fd0f814c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.603 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0fd816700 1 -- 172.21.15.85:0/1337481630 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f400b7f0 con 0x7fd0f80cb760 2022-12-01T09:39:59.603 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0fc814700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd0f80cad00 0x7fd0f814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:39:59.604 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0fd816700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80cb760 0x7fd0f814e3d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009fa0 tx=0x7fd0f000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:59.604 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0e5ffb700 1 -- 172.21.15.85:0/1337481630 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000ddb0 con 0x7fd0f80cb760 2022-12-01T09:39:59.604 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0e5ffb700 1 -- 172.21.15.85:0/1337481630 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f000f040 con 0x7fd0f80cb760 2022-12-01T09:39:59.605 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f814d940 con 0x7fd0f80cb760 2022-12-01T09:39:59.605 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0e5ffb700 1 -- 172.21.15.85:0/1337481630 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0003c60 con 0x7fd0f80cb760 2022-12-01T09:39:59.606 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f80ac670 con 0x7fd0f80cb760 2022-12-01T09:39:59.606 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0e5ffb700 1 -- 172.21.15.85:0/1337481630 <== mon.1 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd0f001c030 con 0x7fd0f80cb760 2022-12-01T09:39:59.606 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.586+0000 7fd0e5ffb700 1 -- 172.21.15.85:0/1337481630 <== mon.1 v2:172.21.15.85:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fd0f00047b0 con 0x7fd0f80cb760 2022-12-01T09:39:59.607 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.587+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8068b70 con 0x7fd0f80cb760 2022-12-01T09:39:59.607 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.589+0000 7fd0e5ffb700 1 -- 172.21.15.85:0/1337481630 <== mon.1 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fd0f001a070 con 0x7fd0f80cb760 2022-12-01T09:39:59.676 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.673+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9f624f12-fff1-44e7-b4f2-76065630d976", "id": 5} v 0) v1 -- 0x7fd0f815c3f0 con 0x7fd0f80cb760 2022-12-01T09:39:59.684 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.681+0000 7fd0e5ffb700 1 -- 172.21.15.85:0/1337481630 <== mon.1 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f624f12-fff1-44e7-b4f2-76065630d976", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd0f0004030 con 0x7fd0f80cb760 2022-12-01T09:39:59.685 INFO:teuthology.orchestra.run.smithi085.stdout:5 2022-12-01T09:39:59.686 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80cb760 msgr2=0x7fd0f814e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.686 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80cb760 0x7fd0f814e3d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009fa0 tx=0x7fd0f000d3b0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.686 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 shutdown_connections 2022-12-01T09:39:59.686 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fd0f80cb760 0x7fd0f814e3d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.687 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0f80b12c0 0x7fd0f814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.687 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 --2- 172.21.15.85:0/1337481630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fd0f80cad00 0x7fd0f814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.687 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 >> 172.21.15.85:0/1337481630 conn(0x7fd0f80cbc00 msgr2=0x7fd0f806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:59.687 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 shutdown_connections 2022-12-01T09:39:59.688 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.682+0000 7fd0ff279700 1 -- 172.21.15.85:0/1337481630 wait complete. 2022-12-01T09:39:59.697 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new ecaf66b0-139a-4a68-8a19-914f382fee4e 6 2022-12-01T09:39:59.823 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.819+0000 7f066cc2c700 1 Processor -- start 2022-12-01T09:39:59.823 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.819+0000 7f066cc2c700 1 -- start start 2022-12-01T09:39:59.826 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f066cc2c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0668069570 0x7f066806c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.826 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f066cc2c700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f066806c770 0x7f066815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.826 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f066cc2c700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06680cde20 con 0x7f066806dad0 2022-12-01T09:39:59.827 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f066cc2c700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066806cf80 con 0x7f0668069570 2022-12-01T09:39:59.827 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f066cc2c700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066815c3c0 con 0x7f066806c770 2022-12-01T09:39:59.827 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f0666d9d700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f066806c770 0x7f066815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.827 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f066659c700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f066806dad0 0x7f0668068e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:48374/0 (socket says 172.21.15.85:48374) 2022-12-01T09:39:59.828 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f066659c700 1 -- 172.21.15.85:0/1482142910 learned_addr learned my addr 172.21.15.85:0/1482142910 (peer_addr_for_me v1:172.21.15.85:0/0) 2022-12-01T09:39:59.828 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.820+0000 7f0665d9b700 1 --2- 172.21.15.85:0/1482142910 >> v2:172.21.15.2:3300/0 conn(0x7f0668069570 0x7f066806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.828 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f0666d9d700 1 -- 172.21.15.85:0/1482142910 >> v2:172.21.15.2:3300/0 conn(0x7f0668069570 msgr2=0x7f066806c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:39:59.828 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f0666d9d700 1 --2- 172.21.15.85:0/1482142910 >> v2:172.21.15.2:3300/0 conn(0x7f0668069570 0x7f066806c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.829 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f0666d9d700 1 -- 172.21.15.85:0/1482142910 >> v1:172.21.15.85:6789/0 conn(0x7f066806dad0 legacy=0x7f0668068e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.829 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f066559a700 1 -- 172.21.15.85:0/1482142910 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006214870 0 0) 0x7f06680cde20 con 0x7f066806dad0 2022-12-01T09:39:59.829 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f0666d9d700 1 -- 172.21.15.85:0/1482142910 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066815c6a0 con 0x7f066806c770 2022-12-01T09:39:59.829 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f0666d9d700 1 --2- 172.21.15.85:0/1482142910 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806c770 0x7f066815f8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f06680cc790 tx=0x7f066000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb7c694dbbdcdc96 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:59.830 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f066559a700 1 -- 172.21.15.85:0/1482142910 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000e070 con 0x7f066806c770 2022-12-01T09:39:59.830 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f066559a700 1 -- 172.21.15.85:0/1482142910 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000bb20 con 0x7f066806c770 2022-12-01T09:39:59.830 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.821+0000 7f066559a700 1 -- 172.21.15.85:0/1482142910 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660012430 con 0x7f066806c770 2022-12-01T09:39:59.830 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.822+0000 7f066cc2c700 1 -- 172.21.15.85:0/1482142910 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806c770 msgr2=0x7f066815f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.831 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.822+0000 7f066cc2c700 1 --2- 172.21.15.85:0/1482142910 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806c770 0x7f066815f8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f06680cc790 tx=0x7f066000b330 comp rx=0 tx=0).stop 2022-12-01T09:39:59.831 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.822+0000 7f066cc2c700 1 -- 172.21.15.85:0/1482142910 shutdown_connections 2022-12-01T09:39:59.831 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.822+0000 7f066cc2c700 1 --2- 172.21.15.85:0/1482142910 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806c770 0x7f066815f8a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.831 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.822+0000 7f066cc2c700 1 --2- 172.21.15.85:0/1482142910 >> v2:172.21.15.2:3300/0 conn(0x7f0668069570 0x7f066806c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.832 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.822+0000 7f066cc2c700 1 -- 172.21.15.85:0/1482142910 >> 172.21.15.85:0/1482142910 conn(0x7f06680cbc00 msgr2=0x7f06680cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:59.832 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.822+0000 7f066cc2c700 1 -- 172.21.15.85:0/1482142910 shutdown_connections 2022-12-01T09:39:59.832 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.822+0000 7f066cc2c700 1 -- 172.21.15.85:0/1482142910 wait complete. 2022-12-01T09:39:59.835 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.823+0000 7f066cc2c700 1 Processor -- start 2022-12-01T09:39:59.836 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.823+0000 7f066cc2c700 1 -- start start 2022-12-01T09:39:59.836 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066cc2c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0668069570 0x7f06680adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.836 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066cc2c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f066806c770 0x7f06680aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.837 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066cc2c700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806dad0 0x7f0668157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:39:59.837 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066cc2c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680af020 con 0x7f066806c770 2022-12-01T09:39:59.837 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066cc2c700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06680af230 con 0x7f066806dad0 2022-12-01T09:39:59.837 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066cc2c700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06680af510 con 0x7f0668069570 2022-12-01T09:39:59.838 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066659c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0668069570 0x7f06680adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.838 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f0666d9d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806dad0 0x7f0668157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.838 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066659c700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0668069570 0x7f06680adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3301/0 says I am v2:172.21.15.85:35456/0 (socket says 172.21.15.85:35456) 2022-12-01T09:39:59.838 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f0666d9d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806dad0 0x7f0668157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42266/0 (socket says 172.21.15.85:42266) 2022-12-01T09:39:59.839 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066659c700 1 -- 172.21.15.85:0/3303739290 learned_addr learned my addr 172.21.15.85:0/3303739290 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:39:59.839 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f0665d9b700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f066806c770 0x7f06680aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:39:59.839 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066659c700 1 -- 172.21.15.85:0/3303739290 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806dad0 msgr2=0x7f0668157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.839 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066659c700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806dad0 0x7f0668157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.840 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066659c700 1 -- 172.21.15.85:0/3303739290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f066806c770 msgr2=0x7f06680aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.840 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066659c700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f066806c770 0x7f06680aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.840 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f066659c700 1 -- 172.21.15.85:0/3303739290 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660009a70 con 0x7f0668069570 2022-12-01T09:39:59.840 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.824+0000 7f0666d9d700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806dad0 0x7f0668157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:39:59.841 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.825+0000 7f066659c700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0668069570 0x7f06680adb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f065800c550 tx=0x7f065800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:39:59.841 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.825+0000 7f0656ffd700 1 -- 172.21.15.85:0/3303739290 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658004c80 con 0x7f0668069570 2022-12-01T09:39:59.841 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.825+0000 7f0656ffd700 1 -- 172.21.15.85:0/3303739290 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0658004e00 con 0x7f0668069570 2022-12-01T09:39:59.841 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.825+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06680af7f0 con 0x7f0668069570 2022-12-01T09:39:59.842 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.825+0000 7f0656ffd700 1 -- 172.21.15.85:0/3303739290 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658007310 con 0x7f0668069570 2022-12-01T09:39:59.842 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.825+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0668157f90 con 0x7f0668069570 2022-12-01T09:39:59.842 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.825+0000 7f0656ffd700 1 -- 172.21.15.85:0/3303739290 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f065801a020 con 0x7f0668069570 2022-12-01T09:39:59.842 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.825+0000 7f0656ffd700 1 -- 172.21.15.85:0/3303739290 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f06580139b0 con 0x7f0668069570 2022-12-01T09:39:59.843 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.826+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06680b2b30 con 0x7f0668069570 2022-12-01T09:39:59.843 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.828+0000 7f0656ffd700 1 -- 172.21.15.85:0/3303739290 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f065801d4a0 con 0x7f0668069570 2022-12-01T09:39:59.856 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:59.853+0000 7feafdd4a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-12-01T09:39:59.907 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:59.904+0000 7f73f316d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-12-01T09:39:59.914 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.911+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ecaf66b0-139a-4a68-8a19-914f382fee4e", "id": 6} v 0) v1 -- 0x7f0668068b70 con 0x7f0668069570 2022-12-01T09:39:59.919 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.916+0000 7f0656ffd700 1 -- 172.21.15.85:0/3303739290 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ecaf66b0-139a-4a68-8a19-914f382fee4e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0658013d20 con 0x7f0668069570 2022-12-01T09:39:59.920 INFO:teuthology.orchestra.run.smithi085.stdout:6 2022-12-01T09:39:59.921 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0668069570 msgr2=0x7f06680adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:39:59.921 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0668069570 0x7f06680adb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f065800c550 tx=0x7f065800c860 comp rx=0 tx=0).stop 2022-12-01T09:39:59.921 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 shutdown_connections 2022-12-01T09:39:59.922 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f066806dad0 0x7f0668157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.922 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0668069570 0x7f06680adb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.922 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 --2- 172.21.15.85:0/3303739290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f066806c770 0x7f06680aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:39:59.922 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 >> 172.21.15.85:0/3303739290 conn(0x7f06680cbc00 msgr2=0x7f06680ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:39:59.923 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 shutdown_connections 2022-12-01T09:39:59.923 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:39:59.917+0000 7f066cc2c700 1 -- 172.21.15.85:0/3303739290 wait complete. 2022-12-01T09:39:59.932 DEBUG:teuthology.orchestra.run.smithi085:> sudo ceph --cluster ceph osd new e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc 7 2022-12-01T09:39:59.936 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:39:59.933+0000 7feafdd4a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-12-01T09:39:59.986 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:39:59.983+0000 7f73f316d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-12-01T09:40:00.053 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.049+0000 7f56038e0700 1 Processor -- start 2022-12-01T09:40:00.053 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.049+0000 7f56038e0700 1 -- start start 2022-12-01T09:40:00.063 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f56038e0700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f55fc0c9220 0x7f55fc0ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.064 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f56038e0700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f55fc0cae00 0x7f55fc06bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.064 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f56038e0700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc0c9820 con 0x7f55fc158be0 2022-12-01T09:40:00.065 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f56038e0700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc06a820 con 0x7f55fc0cae00 2022-12-01T09:40:00.065 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f56038e0700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc0c9b00 con 0x7f55fc0c9220 2022-12-01T09:40:00.065 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f5600e7b700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f55fc0c9220 0x7f55fc0ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.065 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f5600e7b700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f55fc0c9220 0x7f55fc0ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42270/0 (socket says 172.21.15.85:42270) 2022-12-01T09:40:00.066 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f560167c700 1 --1- >> v1:172.21.15.85:6789/0 conn(0x7f55fc158be0 0x7f55fc06a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.85:48386/0 (socket says 172.21.15.85:48386) 2022-12-01T09:40:00.066 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f5600e7b700 1 -- 172.21.15.85:0/4003046841 learned_addr learned my addr 172.21.15.85:0/4003046841 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:40:00.066 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.050+0000 7f5601e7d700 1 --2- 172.21.15.85:0/4003046841 >> v2:172.21.15.2:3300/0 conn(0x7f55fc0cae00 0x7f55fc06bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.066 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f5600e7b700 1 -- 172.21.15.85:0/4003046841 >> v2:172.21.15.2:3300/0 conn(0x7f55fc0cae00 msgr2=0x7f55fc06bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:00.067 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f5600e7b700 1 --2- 172.21.15.85:0/4003046841 >> v2:172.21.15.2:3300/0 conn(0x7f55fc0cae00 0x7f55fc06bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.067 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f5600e7b700 1 -- 172.21.15.85:0/4003046841 >> v1:172.21.15.85:6789/0 conn(0x7f55fc158be0 legacy=0x7f55fc06a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.067 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f55ebfff700 1 -- 172.21.15.85:0/4003046841 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997287009 0 0) 0x7f55fc0c9820 con 0x7f55fc158be0 2022-12-01T09:40:00.067 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f5600e7b700 1 -- 172.21.15.85:0/4003046841 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc0c9de0 con 0x7f55fc0c9220 2022-12-01T09:40:00.068 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f5600e7b700 1 --2- 172.21.15.85:0/4003046841 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0c9220 0x7f55fc0ca8c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0098f0 tx=0x7f55ec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ace233a3bbc416 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:00.068 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f55ebfff700 1 -- 172.21.15.85:0/4003046841 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011830 con 0x7f55fc0c9220 2022-12-01T09:40:00.068 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f55ebfff700 1 -- 172.21.15.85:0/4003046841 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec011990 con 0x7f55fc0c9220 2022-12-01T09:40:00.068 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.051+0000 7f55ebfff700 1 -- 172.21.15.85:0/4003046841 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011d20 con 0x7f55fc0c9220 2022-12-01T09:40:00.069 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.052+0000 7f56038e0700 1 -- 172.21.15.85:0/4003046841 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0c9220 msgr2=0x7f55fc0ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.069 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.052+0000 7f56038e0700 1 --2- 172.21.15.85:0/4003046841 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0c9220 0x7f55fc0ca8c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0098f0 tx=0x7f55ec009c00 comp rx=0 tx=0).stop 2022-12-01T09:40:00.069 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.052+0000 7f56038e0700 1 -- 172.21.15.85:0/4003046841 shutdown_connections 2022-12-01T09:40:00.069 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.052+0000 7f56038e0700 1 --2- 172.21.15.85:0/4003046841 >> v2:172.21.15.2:3300/0 conn(0x7f55fc0cae00 0x7f55fc06bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.070 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.052+0000 7f56038e0700 1 --2- 172.21.15.85:0/4003046841 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0c9220 0x7f55fc0ca8c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.070 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.052+0000 7f56038e0700 1 -- 172.21.15.85:0/4003046841 >> 172.21.15.85:0/4003046841 conn(0x7f55fc0ce8f0 msgr2=0x7f55fc0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:00.070 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.052+0000 7f56038e0700 1 -- 172.21.15.85:0/4003046841 shutdown_connections 2022-12-01T09:40:00.070 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.052+0000 7f56038e0700 1 -- 172.21.15.85:0/4003046841 wait complete. 2022-12-01T09:40:00.070 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.054+0000 7f56038e0700 1 Processor -- start 2022-12-01T09:40:00.071 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.054+0000 7f56038e0700 1 -- start start 2022-12-01T09:40:00.071 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f56038e0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55fc0c9220 0x7f55fc159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.071 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f56038e0700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0cae00 0x7f55fc14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.071 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f56038e0700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55fc158be0 0x7f55fc0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.072 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f56038e0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc14d240 con 0x7f55fc0c9220 2022-12-01T09:40:00.072 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f56038e0700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc153690 con 0x7f55fc0cae00 2022-12-01T09:40:00.072 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f56038e0700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc159f50 con 0x7f55fc158be0 2022-12-01T09:40:00.072 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5600e7b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0cae00 0x7f55fc14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.073 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f560167c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55fc0c9220 0x7f55fc159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.073 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5600e7b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0cae00 0x7f55fc14cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.85:3300/0 says I am v2:172.21.15.85:42276/0 (socket says 172.21.15.85:42276) 2022-12-01T09:40:00.073 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5600e7b700 1 -- 172.21.15.85:0/2170617142 learned_addr learned my addr 172.21.15.85:0/2170617142 (peer_addr_for_me v2:172.21.15.85:0/0) 2022-12-01T09:40:00.073 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5601e7d700 1 --2- 172.21.15.85:0/2170617142 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55fc158be0 0x7f55fc0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.073 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5600e7b700 1 -- 172.21.15.85:0/2170617142 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55fc158be0 msgr2=0x7f55fc0cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.074 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5600e7b700 1 --2- 172.21.15.85:0/2170617142 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55fc158be0 0x7f55fc0cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.074 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5600e7b700 1 -- 172.21.15.85:0/2170617142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55fc0c9220 msgr2=0x7f55fc159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.074 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5600e7b700 1 --2- 172.21.15.85:0/2170617142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55fc0c9220 0x7f55fc159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.074 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.055+0000 7f5600e7b700 1 -- 172.21.15.85:0/2170617142 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec009550 con 0x7f55fc0cae00 2022-12-01T09:40:00.075 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.056+0000 7f5600e7b700 1 --2- 172.21.15.85:0/2170617142 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0cae00 0x7f55fc14cd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55ec014040 tx=0x7f55ec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:00.075 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.056+0000 7f55e9ffb700 1 -- 172.21.15.85:0/2170617142 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004510 con 0x7f55fc0cae00 2022-12-01T09:40:00.075 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.056+0000 7f55e9ffb700 1 -- 172.21.15.85:0/2170617142 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec01e070 con 0x7f55fc0cae00 2022-12-01T09:40:00.075 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.056+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc15a230 con 0x7f55fc0cae00 2022-12-01T09:40:00.076 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.056+0000 7f55e9ffb700 1 -- 172.21.15.85:0/2170617142 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0038c0 con 0x7f55fc0cae00 2022-12-01T09:40:00.076 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.056+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc15a7f0 con 0x7f55fc0cae00 2022-12-01T09:40:00.076 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.056+0000 7f55e9ffb700 1 -- 172.21.15.85:0/2170617142 <== mon.1 v2:172.21.15.85:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f55ec00f050 con 0x7f55fc0cae00 2022-12-01T09:40:00.076 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.056+0000 7f55e9ffb700 1 -- 172.21.15.85:0/2170617142 <== mon.1 v2:172.21.15.85:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f55ec02dd80 con 0x7f55fc0cae00 2022-12-01T09:40:00.077 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.057+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc068b70 con 0x7f55fc0cae00 2022-12-01T09:40:00.077 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.061+0000 7f55e9ffb700 1 -- 172.21.15.85:0/2170617142 <== mon.1 v2:172.21.15.85:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f55ec020070 con 0x7f55fc0cae00 2022-12-01T09:40:00.083 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:00.079+0000 7feafdd4a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-12-01T09:40:00.128 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:00.125+0000 7f73f316d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-12-01T09:40:00.146 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.143+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc", "id": 7} v 0) v1 -- 0x7f55fc15c0f0 con 0x7f55fc0cae00 2022-12-01T09:40:00.154 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.151+0000 7f55e9ffb700 1 -- 172.21.15.85:0/2170617142 <== mon.1 v2:172.21.15.85:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f55ec015020 con 0x7f55fc0cae00 2022-12-01T09:40:00.155 INFO:teuthology.orchestra.run.smithi085.stdout:7 2022-12-01T09:40:00.156 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0cae00 msgr2=0x7f55fc14cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.157 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 --2- 172.21.15.85:0/2170617142 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0cae00 0x7f55fc14cd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55ec014040 tx=0x7f55ec004750 comp rx=0 tx=0).stop 2022-12-01T09:40:00.157 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 shutdown_connections 2022-12-01T09:40:00.157 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 --2- 172.21.15.85:0/2170617142 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f55fc158be0 0x7f55fc0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.157 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 --2- 172.21.15.85:0/2170617142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55fc0c9220 0x7f55fc159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.158 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 --2- 172.21.15.85:0/2170617142 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f55fc0cae00 0x7f55fc14cd00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.158 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 >> 172.21.15.85:0/2170617142 conn(0x7f55fc0ce8f0 msgr2=0x7f55fc064fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:00.158 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 shutdown_connections 2022-12-01T09:40:00.159 INFO:teuthology.orchestra.run.smithi085.stderr:2022-12-01T09:40:00.152+0000 7f56038e0700 1 -- 172.21.15.85:0/2170617142 wait complete. 2022-12-01T09:40:00.168 INFO:tasks.ceph.osd.0:Restarting daemon 2022-12-01T09:40:00.168 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-12-01T09:40:00.171 INFO:tasks.ceph.osd.0:Started 2022-12-01T09:40:00.172 INFO:tasks.ceph.osd.1:Restarting daemon 2022-12-01T09:40:00.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-12-01T09:40:00.174 INFO:tasks.ceph.osd.1:Started 2022-12-01T09:40:00.175 INFO:tasks.ceph.osd.2:Restarting daemon 2022-12-01T09:40:00.175 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-12-01T09:40:00.177 INFO:tasks.ceph.osd.2:Started 2022-12-01T09:40:00.178 INFO:tasks.ceph.osd.3:Restarting daemon 2022-12-01T09:40:00.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-12-01T09:40:00.181 INFO:tasks.ceph.osd.3:Started 2022-12-01T09:40:00.182 INFO:tasks.ceph.osd.4:Restarting daemon 2022-12-01T09:40:00.182 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-12-01T09:40:00.210 INFO:tasks.ceph.osd.4:Started 2022-12-01T09:40:00.211 INFO:tasks.ceph.osd.5:Restarting daemon 2022-12-01T09:40:00.212 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-12-01T09:40:00.214 INFO:tasks.ceph.osd.5:Started 2022-12-01T09:40:00.214 INFO:tasks.ceph.osd.6:Restarting daemon 2022-12-01T09:40:00.215 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-12-01T09:40:00.217 INFO:tasks.ceph.osd.6:Started 2022-12-01T09:40:00.218 INFO:tasks.ceph.osd.7:Restarting daemon 2022-12-01T09:40:00.218 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-12-01T09:40:00.220 INFO:tasks.ceph.osd.7:Started 2022-12-01T09:40:00.221 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-12-01T09:40:00.260 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:00.256+0000 7feafdd4a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-12-01T09:40:00.334 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:00.331+0000 7f73f316d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-12-01T09:40:00.365 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.362+0000 7f43d03f9700 1 Processor -- start 2022-12-01T09:40:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.362+0000 7f43d03f9700 1 -- start start 2022-12-01T09:40:00.368 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.363+0000 7f43d03f9700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 0x7f43c80ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.363+0000 7f43d03f9700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f43c806fae0 0x7f43c806ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.363+0000 7f43d03f9700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c80ca9d0 con 0x7f43c80b1740 2022-12-01T09:40:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43d03f9700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c80704a0 con 0x7f43c806fae0 2022-12-01T09:40:00.370 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43d03f9700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80cacb0 con 0x7f43c8159c80 2022-12-01T09:40:00.370 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce996700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f43c806fae0 0x7f43c806ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.370 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce996700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f43c806fae0 0x7f43c806ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60908/0 (socket says 172.21.15.2:60908) 2022-12-01T09:40:00.370 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce195700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 0x7f43c80ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.371 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce996700 1 -- 172.21.15.2:0/861960220 learned_addr learned my addr 172.21.15.2:0/861960220 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:00.371 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43cd994700 1 --1- 172.21.15.2:0/861960220 >> v1:172.21.15.85:6789/0 conn(0x7f43c80b1740 0x7f43c80b4290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.85:6789/0 says I am v1:172.21.15.2:51212/0 (socket says 172.21.15.2:51212) 2022-12-01T09:40:00.371 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce996700 1 -- 172.21.15.2:0/861960220 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 msgr2=0x7f43c80ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:00.371 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce996700 1 --2- 172.21.15.2:0/861960220 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 0x7f43c80ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.371 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce996700 1 -- 172.21.15.2:0/861960220 >> v1:172.21.15.85:6789/0 conn(0x7f43c80b1740 legacy=0x7f43c80b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.372 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce996700 1 -- 172.21.15.2:0/861960220 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80caf90 con 0x7f43c806fae0 2022-12-01T09:40:00.372 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43cd193700 1 -- 172.21.15.2:0/861960220 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529987481 0 0) 0x7f43c80ca9d0 con 0x7f43c80b1740 2022-12-01T09:40:00.372 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43ce996700 1 --2- 172.21.15.2:0/861960220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c806ff60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009e50 tx=0x7f43c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0ed0ef6f36589c server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:00.372 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43cd193700 1 -- 172.21.15.2:0/861960220 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000e070 con 0x7f43c806fae0 2022-12-01T09:40:00.373 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43cd193700 1 -- 172.21.15.2:0/861960220 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c000b910 con 0x7f43c806fae0 2022-12-01T09:40:00.373 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.364+0000 7f43cd193700 1 -- 172.21.15.2:0/861960220 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000bcc0 con 0x7f43c806fae0 2022-12-01T09:40:00.373 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.365+0000 7f43d03f9700 1 -- 172.21.15.2:0/861960220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 msgr2=0x7f43c806ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.374 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.365+0000 7f43d03f9700 1 --2- 172.21.15.2:0/861960220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c806ff60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009e50 tx=0x7f43c000b330 comp rx=0 tx=0).stop 2022-12-01T09:40:00.374 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.365+0000 7f43d03f9700 1 -- 172.21.15.2:0/861960220 shutdown_connections 2022-12-01T09:40:00.374 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.365+0000 7f43d03f9700 1 --2- 172.21.15.2:0/861960220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c806ff60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.374 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.365+0000 7f43d03f9700 1 --2- 172.21.15.2:0/861960220 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 0x7f43c80ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.374 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.365+0000 7f43d03f9700 1 -- 172.21.15.2:0/861960220 >> 172.21.15.2:0/861960220 conn(0x7f43c80ce8f0 msgr2=0x7f43c80cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:00.375 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.365+0000 7f43d03f9700 1 -- 172.21.15.2:0/861960220 shutdown_connections 2022-12-01T09:40:00.375 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.365+0000 7f43d03f9700 1 -- 172.21.15.2:0/861960220 wait complete. 2022-12-01T09:40:00.378 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.366+0000 7f43d03f9700 1 Processor -- start 2022-12-01T09:40:00.378 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.366+0000 7f43d03f9700 1 -- start start 2022-12-01T09:40:00.379 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43d03f9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c8151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.379 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43d03f9700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f43c80b1740 0x7f43c815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.379 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43d03f9700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 0x7f43c80b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:00.379 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43d03f9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80c5e20 con 0x7f43c806fae0 2022-12-01T09:40:00.380 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43d03f9700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c815d040 con 0x7f43c80b1740 2022-12-01T09:40:00.380 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43d03f9700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80c5af0 con 0x7f43c8159c80 2022-12-01T09:40:00.380 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43ce195700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c8151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.380 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43ce195700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c8151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60910/0 (socket says 172.21.15.2:60910) 2022-12-01T09:40:00.381 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43ce195700 1 -- 172.21.15.2:0/3142121779 learned_addr learned my addr 172.21.15.2:0/3142121779 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:00.381 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43ce996700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 0x7f43c80b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.381 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43cd994700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f43c80b1740 0x7f43c815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:00.381 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43ce195700 1 -- 172.21.15.2:0/3142121779 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 msgr2=0x7f43c80b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.381 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.367+0000 7f43ce195700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 0x7f43c80b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.382 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43ce195700 1 -- 172.21.15.2:0/3142121779 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f43c80b1740 msgr2=0x7f43c815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.382 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43ce195700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f43c80b1740 0x7f43c815c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.382 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43ce195700 1 -- 172.21.15.2:0/3142121779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0009b30 con 0x7f43c806fae0 2022-12-01T09:40:00.383 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43ce195700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c8151b30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9b0 tx=0x7f43c400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:00.383 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43b6ffd700 1 -- 172.21.15.2:0/3142121779 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4009710 con 0x7f43c806fae0 2022-12-01T09:40:00.383 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43b6ffd700 1 -- 172.21.15.2:0/3142121779 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c400ee50 con 0x7f43c806fae0 2022-12-01T09:40:00.383 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c815ca80 con 0x7f43c806fae0 2022-12-01T09:40:00.384 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43b6ffd700 1 -- 172.21.15.2:0/3142121779 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400ca50 con 0x7f43c806fae0 2022-12-01T09:40:00.384 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c815c750 con 0x7f43c806fae0 2022-12-01T09:40:00.384 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43b6ffd700 1 -- 172.21.15.2:0/3142121779 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f43c4019030 con 0x7f43c806fae0 2022-12-01T09:40:00.384 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.368+0000 7f43b6ffd700 1 -- 172.21.15.2:0/3142121779 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f43c4006880 con 0x7f43c806fae0 2022-12-01T09:40:00.385 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.369+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c8068b70 con 0x7f43c806fae0 2022-12-01T09:40:00.385 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.371+0000 7f43b6ffd700 1 -- 172.21.15.2:0/3142121779 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f43c4021060 con 0x7f43c806fae0 2022-12-01T09:40:00.414 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:00.411+0000 7feafdd4a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-12-01T09:40:00.446 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.443+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c8159430 con 0x7f43c806fae0 2022-12-01T09:40:00.447 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.443+0000 7f43b6ffd700 1 -- 172.21.15.2:0/3142121779 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f43c4017070 con 0x7f43c806fae0 2022-12-01T09:40:00.447 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:40:00.448 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"fsid":"3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803","created":"2022-12-01T09:39:56.873720+0000","modified":"2022-12-01T09:40:00.145489+0000","last_up_change":"0.000000","last_in_change":"2022-12-01T09:40:00.145489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"29ad9800-3b0e-4728-9c4b-27eed3978c17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"51463426-88ae-4c28-880b-1b655faff15c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"817f70a8-c4b5-4470-beca-45061a2bf389","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"379bf437-773e-4be5-ac5c-1a8c6b6663e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7ef04bad-dece-4880-9f9c-4ba20e210e0e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9f624f12-fff1-44e7-b4f2-76065630d976","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ecaf66b0-139a-4a68-8a19-914f382fee4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:40:00.449 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 msgr2=0x7f43c8151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:00.449 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c8151b30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9b0 tx=0x7f43c400ecc0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 shutdown_connections 2022-12-01T09:40:00.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f43c8159c80 0x7f43c80b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43c806fae0 0x7f43c8151b30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 --2- 172.21.15.2:0/3142121779 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f43c80b1740 0x7f43c815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:00.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 >> 172.21.15.2:0/3142121779 conn(0x7f43c80ce8f0 msgr2=0x7f43c80cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:00.451 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 shutdown_connections 2022-12-01T09:40:00.451 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:00.444+0000 7f43d03f9700 1 -- 172.21.15.2:0/3142121779 wait complete. 2022-12-01T09:40:00.460 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-12-01T09:40:00.460 INFO:tasks.ceph:Waiting for OSDs to come up 2022-12-01T09:40:00.474 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:00.471+0000 7f73f316d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-12-01T09:40:00.763 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:00.760+0000 7feafdd4a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-12-01T09:40:00.793 INFO:tasks.ceph.osd.0.smithi002.stderr:2022-12-01T09:40:00.789+0000 7f6b073705c0 -1 Falling back to public interface 2022-12-01T09:40:00.803 INFO:tasks.ceph.osd.1.smithi002.stderr:2022-12-01T09:40:00.800+0000 7f59dfb475c0 -1 Falling back to public interface 2022-12-01T09:40:00.804 INFO:tasks.ceph.osd.2.smithi002.stderr:2022-12-01T09:40:00.800+0000 7f956d1f55c0 -1 Falling back to public interface 2022-12-01T09:40:00.824 INFO:tasks.ceph.osd.3.smithi002.stderr:2022-12-01T09:40:00.821+0000 7f6f610d05c0 -1 Falling back to public interface 2022-12-01T09:40:00.825 INFO:tasks.ceph.osd.5.smithi085.stderr:2022-12-01T09:40:00.822+0000 7fa7764f55c0 -1 Falling back to public interface 2022-12-01T09:40:00.826 INFO:tasks.ceph.osd.4.smithi085.stderr:2022-12-01T09:40:00.823+0000 7f4eef92a5c0 -1 Falling back to public interface 2022-12-01T09:40:00.835 INFO:tasks.ceph.osd.7.smithi085.stderr:2022-12-01T09:40:00.832+0000 7f7cceb0c5c0 -1 Falling back to public interface 2022-12-01T09:40:00.842 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:00.839+0000 7f73f316d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-12-01T09:40:00.852 INFO:tasks.ceph.osd.6.smithi085.stderr:2022-12-01T09:40:00.849+0000 7f25ef3cb5c0 -1 Falling back to public interface 2022-12-01T09:40:00.918 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:00.915+0000 7feafdd4a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-12-01T09:40:00.999 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:00.996+0000 7f73f316d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-12-01T09:40:01.030 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:01.027+0000 7feafdd4a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-12-01T09:40:01.119 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:01.116+0000 7f73f316d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-12-01T09:40:01.175 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:01.172+0000 7feafdd4a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-12-01T09:40:01.265 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-12-01T09:40:01.269 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:01.266+0000 7f73f316d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-12-01T09:40:01.272 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:01.269+0000 7feafdd4a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-12-01T09:40:01.375 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:01.372+0000 7f73f316d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-12-01T09:40:01.383 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.379+0000 7fc975a66700 1 Processor -- start 2022-12-01T09:40:01.383 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.379+0000 7fc975a66700 1 -- start start 2022-12-01T09:40:01.386 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.380+0000 7fc975a66700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc97006fae0 0x7fc97006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:01.386 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.380+0000 7fc975a66700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9700ce740 con 0x7fc9700cdaa0 2022-12-01T09:40:01.386 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.380+0000 7fc975a66700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97006bf80 con 0x7fc9700ca0c0 2022-12-01T09:40:01.387 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.380+0000 7fc975a66700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700704a0 con 0x7fc97006fae0 2022-12-01T09:40:01.387 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.380+0000 7fc96effd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fc9700cdaa0 0x7fc9700c8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:33808/0 (socket says 172.21.15.2:33808) 2022-12-01T09:40:01.387 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.380+0000 7fc96effd700 1 -- 172.21.15.2:0/4013734247 learned_addr learned my addr 172.21.15.2:0/4013734247 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:01.388 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.380+0000 7fc96ffff700 1 --2- 172.21.15.2:0/4013734247 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc97006fae0 0x7fc97006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:01.388 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96e7fc700 1 -- 172.21.15.2:0/4013734247 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832705108 0 0) 0x7fc9700ce740 con 0x7fc9700cdaa0 2022-12-01T09:40:01.389 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96e7fc700 1 -- 172.21.15.2:0/4013734247 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc95c0035b0 con 0x7fc9700cdaa0 2022-12-01T09:40:01.389 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96e7fc700 1 -- 172.21.15.2:0/4013734247 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848177111 0 0) 0x7fc97006bf80 con 0x7fc9700ca0c0 2022-12-01T09:40:01.389 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96e7fc700 1 -- 172.21.15.2:0/4013734247 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9700ce740 con 0x7fc9700ca0c0 2022-12-01T09:40:01.389 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96e7fc700 1 -- 172.21.15.2:0/4013734247 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3339260 0 0) 0x7fc95c0035b0 con 0x7fc9700cdaa0 2022-12-01T09:40:01.390 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96ffff700 1 -- 172.21.15.2:0/4013734247 >> v1:172.21.15.85:6789/0 conn(0x7fc9700ca0c0 legacy=0x7fc97006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:01.390 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96ffff700 1 -- 172.21.15.2:0/4013734247 >> v1:172.21.15.2:6789/0 conn(0x7fc9700cdaa0 legacy=0x7fc9700c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:01.390 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96ffff700 1 -- 172.21.15.2:0/4013734247 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9700cdf10 con 0x7fc97006fae0 2022-12-01T09:40:01.390 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96ffff700 1 --2- 172.21.15.2:0/4013734247 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc97006fae0 0x7fc97006ff60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc968009870 tx=0x7fc968009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e068fa3996677972 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:01.391 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96e7fc700 1 -- 172.21.15.2:0/4013734247 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964001670 con 0x7fc97006fae0 2022-12-01T09:40:01.391 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96e7fc700 1 -- 172.21.15.2:0/4013734247 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968004030 con 0x7fc97006fae0 2022-12-01T09:40:01.391 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.381+0000 7fc96e7fc700 1 -- 172.21.15.2:0/4013734247 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9680043c0 con 0x7fc97006fae0 2022-12-01T09:40:01.391 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.382+0000 7fc975a66700 1 -- 172.21.15.2:0/4013734247 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc97006fae0 msgr2=0x7fc97006ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:01.392 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.382+0000 7fc975a66700 1 --2- 172.21.15.2:0/4013734247 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc97006fae0 0x7fc97006ff60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc968009870 tx=0x7fc968009b80 comp rx=0 tx=0).stop 2022-12-01T09:40:01.392 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.382+0000 7fc975a66700 1 -- 172.21.15.2:0/4013734247 shutdown_connections 2022-12-01T09:40:01.392 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.382+0000 7fc975a66700 1 --2- 172.21.15.2:0/4013734247 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc97006fae0 0x7fc97006ff60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:01.392 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.382+0000 7fc975a66700 1 -- 172.21.15.2:0/4013734247 >> 172.21.15.2:0/4013734247 conn(0x7fc9700ce8f0 msgr2=0x7fc9700cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:01.393 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.382+0000 7fc975a66700 1 -- 172.21.15.2:0/4013734247 shutdown_connections 2022-12-01T09:40:01.393 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.382+0000 7fc975a66700 1 -- 172.21.15.2:0/4013734247 wait complete. 2022-12-01T09:40:01.396 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.384+0000 7fc975a66700 1 Processor -- start 2022-12-01T09:40:01.396 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.384+0000 7fc975a66700 1 -- start start 2022-12-01T09:40:01.397 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc975a66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc97006fae0 0x7fc9700b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:01.397 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc975a66700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc9700ca0c0 0x7fc9700b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:01.397 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc975a66700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc9700cdaa0 0x7fc97015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:01.398 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc975a66700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b3b20 con 0x7fc97006fae0 2022-12-01T09:40:01.399 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc975a66700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b2f10 con 0x7fc9700ca0c0 2022-12-01T09:40:01.400 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc975a66700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9700b31f0 con 0x7fc9700cdaa0 2022-12-01T09:40:01.400 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc97006fae0 0x7fc9700b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:01.400 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc97006fae0 0x7fc9700b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60922/0 (socket says 172.21.15.2:60922) 2022-12-01T09:40:01.400 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 -- 172.21.15.2:0/3479861588 learned_addr learned my addr 172.21.15.2:0/3479861588 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:01.401 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96effd700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc9700ca0c0 0x7fc9700b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:01.401 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96ffff700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc9700cdaa0 0x7fc97015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:01.401 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 -- 172.21.15.2:0/3479861588 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc9700cdaa0 msgr2=0x7fc97015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:01.401 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc9700cdaa0 0x7fc97015cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:01.401 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 -- 172.21.15.2:0/3479861588 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc9700ca0c0 msgr2=0x7fc9700b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:01.402 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc9700ca0c0 0x7fc9700b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:01.402 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 -- 172.21.15.2:0/3479861588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680094a0 con 0x7fc97006fae0 2022-12-01T09:40:01.402 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc96f7fe700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc97006fae0 0x7fc9700b2130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc96000cc50 tx=0x7fc96000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:01.403 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.385+0000 7fc957fff700 1 -- 172.21.15.2:0/3479861588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960003c00 con 0x7fc97006fae0 2022-12-01T09:40:01.403 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.386+0000 7fc957fff700 1 -- 172.21.15.2:0/3479861588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc960003d60 con 0x7fc97006fae0 2022-12-01T09:40:01.403 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.386+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9700b34d0 con 0x7fc97006fae0 2022-12-01T09:40:01.403 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.386+0000 7fc957fff700 1 -- 172.21.15.2:0/3479861588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96000ab10 con 0x7fc97006fae0 2022-12-01T09:40:01.404 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.386+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9700b2670 con 0x7fc97006fae0 2022-12-01T09:40:01.404 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.386+0000 7fc957fff700 1 -- 172.21.15.2:0/3479861588 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc960010050 con 0x7fc97006fae0 2022-12-01T09:40:01.404 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.386+0000 7fc957fff700 1 -- 172.21.15.2:0/3479861588 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fc960012af0 con 0x7fc97006fae0 2022-12-01T09:40:01.404 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.386+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9700733e0 con 0x7fc97006fae0 2022-12-01T09:40:01.405 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.389+0000 7fc957fff700 1 -- 172.21.15.2:0/3479861588 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fc960012dd0 con 0x7fc97006fae0 2022-12-01T09:40:01.463 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.460+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc970068b70 con 0x7fc97006fae0 2022-12-01T09:40:01.464 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.461+0000 7fc957fff700 1 -- 172.21.15.2:0/3479861588 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7fc9600050d0 con 0x7fc97006fae0 2022-12-01T09:40:01.465 INFO:teuthology.misc.health.smithi002.stdout: 2022-12-01T09:40:01.465 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":10,"fsid":"3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803","created":"2022-12-01T09:39:56.873720+0000","modified":"2022-12-01T09:40:00.145489+0000","last_up_change":"0.000000","last_in_change":"2022-12-01T09:40:00.145489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"29ad9800-3b0e-4728-9c4b-27eed3978c17","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"51463426-88ae-4c28-880b-1b655faff15c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"817f70a8-c4b5-4470-beca-45061a2bf389","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"379bf437-773e-4be5-ac5c-1a8c6b6663e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7ef04bad-dece-4880-9f9c-4ba20e210e0e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9f624f12-fff1-44e7-b4f2-76065630d976","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ecaf66b0-139a-4a68-8a19-914f382fee4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:40:01.466 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc97006fae0 msgr2=0x7fc9700b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:01.466 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc97006fae0 0x7fc9700b2130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc96000cc50 tx=0x7fc96000cf60 comp rx=0 tx=0).stop 2022-12-01T09:40:01.467 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 shutdown_connections 2022-12-01T09:40:01.467 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fc9700cdaa0 0x7fc97015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:01.467 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc97006fae0 0x7fc9700b2130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:01.467 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 --2- 172.21.15.2:0/3479861588 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fc9700ca0c0 0x7fc9700b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:01.468 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 >> 172.21.15.2:0/3479861588 conn(0x7fc9700ce8f0 msgr2=0x7fc97015fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:01.468 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 shutdown_connections 2022-12-01T09:40:01.468 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:01.462+0000 7fc975a66700 1 -- 172.21.15.2:0/3479861588 wait complete. 2022-12-01T09:40:01.477 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-12-01T09:40:01.623 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:01.619+0000 7feafdd4a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-12-01T09:40:01.745 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:01.741+0000 7f73f316d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-12-01T09:40:02.202 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:02.198+0000 7feafdd4a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-12-01T09:40:02.353 INFO:tasks.ceph.mgr.y.smithi002.stderr:2022-12-01T09:40:02.350+0000 7f73f316d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-12-01T09:40:03.231 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:03.227+0000 7feab60fc700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-12-01T09:40:03.232 INFO:tasks.ceph.mgr.x.smithi085.stderr:2022-12-01T09:40:03.228+0000 7feab60fc700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-12-01T09:40:04.123 INFO:tasks.ceph.osd.7.smithi085.stderr:2022-12-01T09:40:04.120+0000 7f7cceb0c5c0 -1 osd.7 0 log_to_monitors true 2022-12-01T09:40:04.290 INFO:tasks.ceph.osd.1.smithi002.stderr:2022-12-01T09:40:04.286+0000 7f59dfb475c0 -1 osd.1 0 log_to_monitors true 2022-12-01T09:40:04.347 INFO:tasks.ceph.osd.3.smithi002.stderr:2022-12-01T09:40:04.344+0000 7f6f610d05c0 -1 osd.3 0 log_to_monitors true 2022-12-01T09:40:04.353 INFO:tasks.ceph.osd.0.smithi002.stderr:2022-12-01T09:40:04.349+0000 7f6b073705c0 -1 osd.0 0 log_to_monitors true 2022-12-01T09:40:04.353 INFO:tasks.ceph.osd.2.smithi002.stderr:2022-12-01T09:40:04.350+0000 7f956d1f55c0 -1 osd.2 0 log_to_monitors true 2022-12-01T09:40:04.367 INFO:tasks.ceph.osd.5.smithi085.stderr:2022-12-01T09:40:04.364+0000 7fa7764f55c0 -1 osd.5 0 log_to_monitors true 2022-12-01T09:40:04.375 INFO:tasks.ceph.osd.4.smithi085.stderr:2022-12-01T09:40:04.372+0000 7f4eef92a5c0 -1 osd.4 0 log_to_monitors true 2022-12-01T09:40:04.378 INFO:tasks.ceph.osd.6.smithi085.stderr:2022-12-01T09:40:04.375+0000 7f25ef3cb5c0 -1 osd.6 0 log_to_monitors true 2022-12-01T09:40:06.124 INFO:tasks.ceph.osd.7.smithi085.stderr:2022-12-01T09:40:06.120+0000 7f7cc26c3700 -1 osd.7 0 waiting for initial osdmap 2022-12-01T09:40:06.143 INFO:tasks.ceph.osd.7.smithi085.stderr:2022-12-01T09:40:06.140+0000 7f7cbaa9e700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:40:07.470 INFO:tasks.ceph.osd.0.smithi002.stderr:2022-12-01T09:40:07.467+0000 7f6afaf27700 -1 osd.0 0 waiting for initial osdmap 2022-12-01T09:40:07.487 INFO:tasks.ceph.osd.0.smithi002.stderr:2022-12-01T09:40:07.484+0000 7f6af3302700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:40:07.540 INFO:tasks.ceph.osd.3.smithi002.stderr:2022-12-01T09:40:07.537+0000 7f6f54c87700 -1 osd.3 0 waiting for initial osdmap 2022-12-01T09:40:07.554 INFO:tasks.ceph.osd.3.smithi002.stderr:2022-12-01T09:40:07.551+0000 7f6f4d062700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:40:07.572 INFO:tasks.ceph.osd.1.smithi002.stderr:2022-12-01T09:40:07.569+0000 7f59d36fe700 -1 osd.1 0 waiting for initial osdmap 2022-12-01T09:40:07.588 INFO:tasks.ceph.osd.1.smithi002.stderr:2022-12-01T09:40:07.585+0000 7f59cbad9700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:40:07.611 INFO:tasks.ceph.osd.2.smithi002.stderr:2022-12-01T09:40:07.607+0000 7f9560dac700 -1 osd.2 0 waiting for initial osdmap 2022-12-01T09:40:07.620 INFO:tasks.ceph.osd.2.smithi002.stderr:2022-12-01T09:40:07.617+0000 7f9559187700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:40:07.911 INFO:tasks.ceph.osd.6.smithi085.stderr:2022-12-01T09:40:07.907+0000 7f25e2f82700 -1 osd.6 0 waiting for initial osdmap 2022-12-01T09:40:07.918 INFO:tasks.ceph.osd.5.smithi085.stderr:2022-12-01T09:40:07.914+0000 7fa76a0ac700 -1 osd.5 0 waiting for initial osdmap 2022-12-01T09:40:07.927 INFO:tasks.ceph.osd.6.smithi085.stderr:2022-12-01T09:40:07.924+0000 7f25db35d700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:40:07.931 INFO:tasks.ceph.osd.5.smithi085.stderr:2022-12-01T09:40:07.928+0000 7fa762487700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:40:07.939 INFO:tasks.ceph.osd.4.smithi085.stderr:2022-12-01T09:40:07.936+0000 7f4ee34e1700 -1 osd.4 0 waiting for initial osdmap 2022-12-01T09:40:07.956 INFO:tasks.ceph.osd.4.smithi085.stderr:2022-12-01T09:40:07.953+0000 7f4edb8bc700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:40:08.284 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-12-01T09:40:08.407 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.403+0000 7f8dc3769700 1 Processor -- start 2022-12-01T09:40:08.407 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.404+0000 7f8dc3769700 1 -- start start 2022-12-01T09:40:08.413 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.404+0000 7f8dc3769700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc069570 0x7f8dbc166e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:08.413 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.404+0000 7f8dc3769700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc0714e0 con 0x7f8dbc15c070 2022-12-01T09:40:08.414 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.404+0000 7f8dc3769700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dbc15c440 con 0x7f8dbc0702f0 2022-12-01T09:40:08.414 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.404+0000 7f8dc3769700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0708d0 con 0x7f8dbc069570 2022-12-01T09:40:08.414 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dc1505700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8dbc15c070 0x7f8dbc0cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:33962/0 (socket says 172.21.15.2:33962) 2022-12-01T09:40:08.414 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dc1505700 1 -- 172.21.15.2:0/122553630 learned_addr learned my addr 172.21.15.2:0/122553630 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:08.414 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dc1d06700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc069570 0x7f8dbc166e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:08.415 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749491324 0 0) 0x7f8dbc0714e0 con 0x7f8dbc15c070 2022-12-01T09:40:08.415 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da40035b0 con 0x7f8dbc15c070 2022-12-01T09:40:08.415 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160998284 0 0) 0x7f8dbc15c440 con 0x7f8dbc0702f0 2022-12-01T09:40:08.415 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dbc0714e0 con 0x7f8dbc0702f0 2022-12-01T09:40:08.416 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3913567308 0 0) 0x7f8da40035b0 con 0x7f8dbc15c070 2022-12-01T09:40:08.416 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc069570 msgr2=0x7f8dbc166e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:08.416 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc069570 0x7f8dbc166e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.416 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 >> v1:172.21.15.85:6789/0 conn(0x7f8dbc0702f0 legacy=0x7f8dbc068e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:08.416 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc070bb0 con 0x7f8dbc15c070 2022-12-01T09:40:08.417 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2890946737 0 0) 0x7f8db8001670 con 0x7f8dbc15c070 2022-12-01T09:40:08.417 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 >> v1:172.21.15.2:6789/0 conn(0x7f8dbc15c070 legacy=0x7f8dbc0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:08.417 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 0x7f8da4004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:08.417 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8da4004850 0x7f8da4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:08.418 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc069570 0x7f8da40052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:08.418 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0708d0 con 0x7f8da4004850 2022-12-01T09:40:08.418 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da400bc00 con 0x7f8da4003c30 2022-12-01T09:40:08.418 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da400bdc0 con 0x7f8dbc069570 2022-12-01T09:40:08.418 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dc1505700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 0x7f8da4004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:08.419 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dc0d04700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8da4004850 0x7f8da4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:08.419 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.405+0000 7f8dc1d06700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc069570 0x7f8da40052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:08.419 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc1505700 1 -- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc069570 msgr2=0x7f8da40052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:08.419 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc1505700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc069570 0x7f8da40052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.420 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc1505700 1 -- 172.21.15.2:0/122553630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8da4004850 msgr2=0x7f8da4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:08.420 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc1505700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8da4004850 0x7f8da4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.420 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc1505700 1 -- 172.21.15.2:0/122553630 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da40035b0 con 0x7f8da4003c30 2022-12-01T09:40:08.420 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc0d04700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8da4004850 0x7f8da4004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:40:08.420 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc1505700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 0x7f8da4004020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c0e0 tx=0x7f8da4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e116d67fc317ec server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:08.421 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800cc80 con 0x7f8da4003c30 2022-12-01T09:40:08.421 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db800cde0 con 0x7f8da4003c30 2022-12-01T09:40:08.421 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dabfff700 1 -- 172.21.15.2:0/122553630 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db8005ba0 con 0x7f8da4003c30 2022-12-01T09:40:08.421 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc3769700 1 -- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 msgr2=0x7f8da4004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:08.422 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc3769700 1 --2- 172.21.15.2:0/122553630 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 0x7f8da4004020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c0e0 tx=0x7f8da4005880 comp rx=0 tx=0).stop 2022-12-01T09:40:08.422 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc1d06700 1 -- 172.21.15.2:0/122553630 reap_dead start 2022-12-01T09:40:08.422 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc3769700 1 -- 172.21.15.2:0/122553630 shutdown_connections 2022-12-01T09:40:08.422 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.406+0000 7f8dc3769700 1 -- 172.21.15.2:0/122553630 >> 172.21.15.2:0/122553630 conn(0x7f8dbc0cc850 msgr2=0x7f8dbc0cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:08.423 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.407+0000 7f8dc3769700 1 -- 172.21.15.2:0/122553630 shutdown_connections 2022-12-01T09:40:08.423 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.407+0000 7f8dc3769700 1 -- 172.21.15.2:0/122553630 wait complete. 2022-12-01T09:40:08.423 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.408+0000 7f8dc3769700 1 Processor -- start 2022-12-01T09:40:08.423 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.408+0000 7f8dc3769700 1 -- start start 2022-12-01T09:40:08.424 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc3769700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 0x7f8dbc1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:08.424 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc3769700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dbc0c6250 0x7f8dbc159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:08.424 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc3769700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc0c66c0 0x7f8dbc15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:08.424 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc3769700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0b3140 con 0x7f8dbc0c6250 2022-12-01T09:40:08.425 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc3769700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0c6c30 con 0x7f8da4003c30 2022-12-01T09:40:08.425 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc3769700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc0c6f10 con 0x7f8dbc0c66c0 2022-12-01T09:40:08.425 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dbc0c6250 0x7f8dbc159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:08.425 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dbc0c6250 0x7f8dbc159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32854/0 (socket says 172.21.15.2:32854) 2022-12-01T09:40:08.426 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 -- 172.21.15.2:0/3229485513 learned_addr learned my addr 172.21.15.2:0/3229485513 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:08.426 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc1505700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 0x7f8dbc1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:08.426 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc1d06700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc0c66c0 0x7f8dbc15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:08.426 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 -- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc0c66c0 msgr2=0x7f8dbc15a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:08.427 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc0c66c0 0x7f8dbc15a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.427 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 -- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 msgr2=0x7f8dbc1533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:08.427 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 0x7f8dbc1533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.427 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 -- 172.21.15.2:0/3229485513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0c71f0 con 0x7f8dbc0c6250 2022-12-01T09:40:08.428 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc0d04700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dbc0c6250 0x7f8dbc159d50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00fb10 tx=0x7f8dac00fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:08.428 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8da9ffb700 1 -- 172.21.15.2:0/3229485513 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0052e0 con 0x7f8dbc0c6250 2022-12-01T09:40:08.428 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8da9ffb700 1 -- 172.21.15.2:0/3229485513 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac005440 con 0x7f8dbc0c6250 2022-12-01T09:40:08.428 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc0c74d0 con 0x7f8dbc0c6250 2022-12-01T09:40:08.428 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.409+0000 7f8da9ffb700 1 -- 172.21.15.2:0/3229485513 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0038c0 con 0x7f8dbc0c6250 2022-12-01T09:40:08.429 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.410+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc1592c0 con 0x7f8dbc0c6250 2022-12-01T09:40:08.430 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.411+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc0b3420 con 0x7f8dbc0c6250 2022-12-01T09:40:08.430 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.411+0000 7f8da9ffb700 1 -- 172.21.15.2:0/3229485513 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7f8dac003a20 con 0x7f8dbc0c6250 2022-12-01T09:40:08.430 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.411+0000 7f8da9ffb700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f8d9c066540 0x7f8d9c0689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:08.431 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.411+0000 7f8da9ffb700 1 -- 172.21.15.2:0/3229485513 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3627+0+0 (secure 0 0 0) 0x7f8dac082f00 con 0x7f8dbc0c6250 2022-12-01T09:40:08.431 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.412+0000 7f8dc1505700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f8d9c066540 0x7f8d9c0689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:08.431 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.412+0000 7f8dc1505700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f8d9c066540 0x7f8d9c0689f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c0e0 tx=0x7f8db8007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:08.431 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.416+0000 7f8da9ffb700 1 -- 172.21.15.2:0/3229485513 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f8dac050fe0 con 0x7f8dbc0c6250 2022-12-01T09:40:08.611 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.607+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dbc065e10 con 0x7f8dbc0c6250 2022-12-01T09:40:08.612 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.608+0000 7f8da9ffb700 1 -- 172.21.15.2:0/3229485513 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9139 (secure 0 0 0) 0x7f8dac003cb0 con 0x7f8dbc0c6250 2022-12-01T09:40:08.613 INFO:teuthology.misc.health.smithi002.stdout: 2022-12-01T09:40:08.613 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":14,"fsid":"3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803","created":"2022-12-01T09:39:56.873720+0000","modified":"2022-12-01T09:40:07.634990+0000","last_up_change":"2022-12-01T09:40:07.634990+0000","last_in_change":"2022-12-01T09:40:00.145489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"29ad9800-3b0e-4728-9c4b-27eed3978c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":101414},{"type":"v1","addr":"172.21.15.2:6801","nonce":101414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":101414},{"type":"v1","addr":"172.21.15.2:6803","nonce":101414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":101414},{"type":"v1","addr":"172.21.15.2:6807","nonce":101414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":101414},{"type":"v1","addr":"172.21.15.2:6805","nonce":101414}]},"public_addr":"172.21.15.2:6801/101414","cluster_addr":"172.21.15.2:6803/101414","heartbeat_back_addr":"172.21.15.2:6807/101414","heartbeat_front_addr":"172.21.15.2:6805/101414","state":["exists","up"]},{"osd":1,"uuid":"51463426-88ae-4c28-880b-1b655faff15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":101416},{"type":"v1","addr":"172.21.15.2:6810","nonce":101416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":101416},{"type":"v1","addr":"172.21.15.2:6814","nonce":101416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":101416},{"type":"v1","addr":"172.21.15.2:6822","nonce":101416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":101416},{"type":"v1","addr":"172.21.15.2:6818","nonce":101416}]},"public_addr":"172.21.15.2:6810/101416","cluster_addr":"172.21.15.2:6814/101416","heartbeat_back_addr":"172.21.15.2:6822/101416","heartbeat_front_addr":"172.21.15.2:6818/101416","state":["exists","up"]},{"osd":2,"uuid":"817f70a8-c4b5-4470-beca-45061a2bf389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6809","nonce":101415},{"type":"v1","addr":"172.21.15.2:6811","nonce":101415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6813","nonce":101415},{"type":"v1","addr":"172.21.15.2:6815","nonce":101415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6821","nonce":101415},{"type":"v1","addr":"172.21.15.2:6823","nonce":101415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6817","nonce":101415},{"type":"v1","addr":"172.21.15.2:6819","nonce":101415}]},"public_addr":"172.21.15.2:6811/101415","cluster_addr":"172.21.15.2:6815/101415","heartbeat_back_addr":"172.21.15.2:6823/101415","heartbeat_front_addr":"172.21.15.2:6819/101415","state":["exists","up"]},{"osd":3,"uuid":"379bf437-773e-4be5-ac5c-1a8c6b6663e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":101458},{"type":"v1","addr":"172.21.15.2:6825","nonce":101458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":101458},{"type":"v1","addr":"172.21.15.2:6827","nonce":101458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":101458},{"type":"v1","addr":"172.21.15.2:6831","nonce":101458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":101458},{"type":"v1","addr":"172.21.15.2:6829","nonce":101458}]},"public_addr":"172.21.15.2:6825/101458","cluster_addr":"172.21.15.2:6827/101458","heartbeat_back_addr":"172.21.15.2:6831/101458","heartbeat_front_addr":"172.21.15.2:6829/101458","state":["exists","up"]},{"osd":4,"uuid":"7ef04bad-dece-4880-9f9c-4ba20e210e0e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9f624f12-fff1-44e7-b4f2-76065630d976","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ecaf66b0-139a-4a68-8a19-914f382fee4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6816","nonce":101673},{"type":"v1","addr":"172.21.15.85:6817","nonce":101673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":101673},{"type":"v1","addr":"172.21.15.85:6819","nonce":101673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":101673},{"type":"v1","addr":"172.21.15.85:6823","nonce":101673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":101673},{"type":"v1","addr":"172.21.15.85:6821","nonce":101673}]},"public_addr":"172.21.15.85:6817/101673","cluster_addr":"172.21.15.85:6819/101673","heartbeat_back_addr":"172.21.15.85:6823/101673","heartbeat_front_addr":"172.21.15.85:6821/101673","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.145226+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:40:08.614 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f8d9c066540 msgr2=0x7f8d9c0689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:08.615 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f8d9c066540 0x7f8d9c0689f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c0e0 tx=0x7f8db8007000 comp rx=0 tx=0).stop 2022-12-01T09:40:08.615 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dbc0c6250 msgr2=0x7f8dbc159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:08.615 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dbc0c6250 0x7f8dbc159d50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00fb10 tx=0x7f8dac00fe20 comp rx=0 tx=0).stop 2022-12-01T09:40:08.615 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 shutdown_connections 2022-12-01T09:40:08.616 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f8d9c066540 0x7f8d9c0689f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.616 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f8dbc0c66c0 0x7f8dbc15a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.616 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f8da4003c30 0x7f8dbc1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.617 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 --2- 172.21.15.2:0/3229485513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dbc0c6250 0x7f8dbc159d50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:08.617 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 >> 172.21.15.2:0/3229485513 conn(0x7f8dbc0cc850 msgr2=0x7f8dbc06c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:08.617 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 shutdown_connections 2022-12-01T09:40:08.617 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:08.610+0000 7f8dc3769700 1 -- 172.21.15.2:0/3229485513 wait complete. 2022-12-01T09:40:08.625 DEBUG:teuthology.misc:5 of 8 OSDs are up 2022-12-01T09:40:15.432 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-12-01T09:40:15.561 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.557+0000 7f3ad40af700 1 Processor -- start 2022-12-01T09:40:15.562 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.558+0000 7f3ad40af700 1 -- start start 2022-12-01T09:40:15.570 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.558+0000 7f3ad40af700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 0x7f3acc0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.571 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.558+0000 7f3ad40af700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3acc0c8d70 0x7f3acc0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.571 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.558+0000 7f3ad40af700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3acc0cc100 con 0x7f3acc0ad460 2022-12-01T09:40:15.571 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.558+0000 7f3ad40af700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc06b470 con 0x7f3acc0c8d70 2022-12-01T09:40:15.572 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.558+0000 7f3ad40af700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0c91f0 con 0x7f3acc070bb0 2022-12-01T09:40:15.572 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad264c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3acc0c8d70 0x7f3acc0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.573 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad1e4b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3acc0ad460 0x7f3acc06fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34062/0 (socket says 172.21.15.2:34062) 2022-12-01T09:40:15.573 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad1e4b700 1 -- 172.21.15.2:0/334958240 learned_addr learned my addr 172.21.15.2:0/334958240 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:15.574 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad264c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3acc0c8d70 0x7f3acc0cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32944/0 (socket says 172.21.15.2:32944) 2022-12-01T09:40:15.574 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad164a700 1 --2- 172.21.15.2:0/334958240 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 0x7f3acc0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.575 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad264c700 1 -- 172.21.15.2:0/334958240 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 msgr2=0x7f3acc0c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:15.575 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad264c700 1 --2- 172.21.15.2:0/334958240 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 0x7f3acc0c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.576 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad264c700 1 -- 172.21.15.2:0/334958240 >> v1:172.21.15.2:6789/0 conn(0x7f3acc0ad460 legacy=0x7f3acc06fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.576 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad264c700 1 -- 172.21.15.2:0/334958240 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc0cbdb0 con 0x7f3acc0c8d70 2022-12-01T09:40:15.576 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad0e49700 1 -- 172.21.15.2:0/334958240 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620588349 0 0) 0x7f3acc0cc100 con 0x7f3acc0ad460 2022-12-01T09:40:15.577 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad264c700 1 --2- 172.21.15.2:0/334958240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0c8d70 0x7f3acc0cc590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009d90 tx=0x7f3ac400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2748c2dd9a95bd server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:15.577 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad0e49700 1 -- 172.21.15.2:0/334958240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400e070 con 0x7f3acc0c8d70 2022-12-01T09:40:15.578 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad0e49700 1 -- 172.21.15.2:0/334958240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac400bbd0 con 0x7f3acc0c8d70 2022-12-01T09:40:15.578 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.559+0000 7f3ad0e49700 1 -- 172.21.15.2:0/334958240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4012430 con 0x7f3acc0c8d70 2022-12-01T09:40:15.579 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.560+0000 7f3ad40af700 1 -- 172.21.15.2:0/334958240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0c8d70 msgr2=0x7f3acc0cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.579 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.560+0000 7f3ad40af700 1 --2- 172.21.15.2:0/334958240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0c8d70 0x7f3acc0cc590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4009d90 tx=0x7f3ac400b330 comp rx=0 tx=0).stop 2022-12-01T09:40:15.579 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.560+0000 7f3ad40af700 1 -- 172.21.15.2:0/334958240 shutdown_connections 2022-12-01T09:40:15.579 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.560+0000 7f3ad40af700 1 --2- 172.21.15.2:0/334958240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0c8d70 0x7f3acc0cc590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.580 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.560+0000 7f3ad40af700 1 --2- 172.21.15.2:0/334958240 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 0x7f3acc0c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.580 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.560+0000 7f3ad40af700 1 -- 172.21.15.2:0/334958240 >> 172.21.15.2:0/334958240 conn(0x7f3acc069aa0 msgr2=0x7f3acc0ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:15.580 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.560+0000 7f3ad40af700 1 -- 172.21.15.2:0/334958240 shutdown_connections 2022-12-01T09:40:15.580 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.560+0000 7f3ad40af700 1 -- 172.21.15.2:0/334958240 wait complete. 2022-12-01T09:40:15.581 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.561+0000 7f3ad40af700 1 Processor -- start 2022-12-01T09:40:15.581 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.561+0000 7f3ad40af700 1 -- start start 2022-12-01T09:40:15.581 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad40af700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 0x7f3acc0ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.581 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad40af700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0ad460 0x7f3acc0c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.582 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad40af700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3acc0c8d70 0x7f3acc15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.582 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad40af700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0c7e10 con 0x7f3acc0ad460 2022-12-01T09:40:15.582 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad40af700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0c7290 con 0x7f3acc0c8d70 2022-12-01T09:40:15.582 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad40af700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0c7570 con 0x7f3acc070bb0 2022-12-01T09:40:15.583 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad164a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0ad460 0x7f3acc0c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.583 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad164a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0ad460 0x7f3acc0c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32948/0 (socket says 172.21.15.2:32948) 2022-12-01T09:40:15.583 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad164a700 1 -- 172.21.15.2:0/217917942 learned_addr learned my addr 172.21.15.2:0/217917942 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:15.583 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.562+0000 7f3ad264c700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3acc0c8d70 0x7f3acc15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.584 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad1e4b700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 0x7f3acc0ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.584 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad164a700 1 -- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 msgr2=0x7f3acc0ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.584 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad164a700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 0x7f3acc0ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.584 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad164a700 1 -- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3acc0c8d70 msgr2=0x7f3acc15b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.585 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad164a700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3acc0c8d70 0x7f3acc15b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.585 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad164a700 1 -- 172.21.15.2:0/217917942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac4009a70 con 0x7f3acc0ad460 2022-12-01T09:40:15.585 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad264c700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3acc0c8d70 0x7f3acc15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:40:15.585 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad164a700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0ad460 0x7f3acc0c67d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000d800 tx=0x7f3ac000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:15.586 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3abe7fc700 1 -- 172.21.15.2:0/217917942 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00125f0 con 0x7f3acc0ad460 2022-12-01T09:40:15.586 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3abe7fc700 1 -- 172.21.15.2:0/217917942 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac0012750 con 0x7f3acc0ad460 2022-12-01T09:40:15.586 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc0c7850 con 0x7f3acc0ad460 2022-12-01T09:40:15.586 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3abe7fc700 1 -- 172.21.15.2:0/217917942 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0012b00 con 0x7f3acc0ad460 2022-12-01T09:40:15.586 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.563+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc14ad50 con 0x7f3acc0ad460 2022-12-01T09:40:15.587 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.564+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc068b70 con 0x7f3acc0ad460 2022-12-01T09:40:15.587 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.564+0000 7f3abe7fc700 1 -- 172.21.15.2:0/217917942 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f3ac0012c60 con 0x7f3acc0ad460 2022-12-01T09:40:15.587 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.565+0000 7f3abe7fc700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f3aac066590 0x7f3aac068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.588 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.565+0000 7f3abe7fc700 1 -- 172.21.15.2:0/217917942 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3ac00837e0 con 0x7f3acc0ad460 2022-12-01T09:40:15.588 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.565+0000 7f3ad1e4b700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f3aac066590 0x7f3aac068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.588 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.566+0000 7f3ad1e4b700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f3aac066590 0x7f3aac068a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8000f60 tx=0x7f3ac8007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:15.589 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.570+0000 7f3abe7fc700 1 -- 172.21.15.2:0/217917942 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f3ac00513a0 con 0x7f3acc0ad460 2022-12-01T09:40:15.767 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.764+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3acc098a50 con 0x7f3acc0ad460 2022-12-01T09:40:15.768 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.764+0000 7f3abe7fc700 1 -- 172.21.15.2:0/217917942 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12125 (secure 0 0 0) 0x7f3ac00549c0 con 0x7f3acc0ad460 2022-12-01T09:40:15.768 INFO:teuthology.misc.health.smithi002.stdout: 2022-12-01T09:40:15.769 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":17,"fsid":"3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803","created":"2022-12-01T09:39:56.873720+0000","modified":"2022-12-01T09:40:10.920497+0000","last_up_change":"2022-12-01T09:40:08.916618+0000","last_in_change":"2022-12-01T09:40:00.145489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-12-01T09:40:08.277187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"29ad9800-3b0e-4728-9c4b-27eed3978c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":101414},{"type":"v1","addr":"172.21.15.2:6801","nonce":101414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":101414},{"type":"v1","addr":"172.21.15.2:6803","nonce":101414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":101414},{"type":"v1","addr":"172.21.15.2:6807","nonce":101414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":101414},{"type":"v1","addr":"172.21.15.2:6805","nonce":101414}]},"public_addr":"172.21.15.2:6801/101414","cluster_addr":"172.21.15.2:6803/101414","heartbeat_back_addr":"172.21.15.2:6807/101414","heartbeat_front_addr":"172.21.15.2:6805/101414","state":["exists","up"]},{"osd":1,"uuid":"51463426-88ae-4c28-880b-1b655faff15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":101416},{"type":"v1","addr":"172.21.15.2:6810","nonce":101416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":101416},{"type":"v1","addr":"172.21.15.2:6814","nonce":101416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":101416},{"type":"v1","addr":"172.21.15.2:6822","nonce":101416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":101416},{"type":"v1","addr":"172.21.15.2:6818","nonce":101416}]},"public_addr":"172.21.15.2:6810/101416","cluster_addr":"172.21.15.2:6814/101416","heartbeat_back_addr":"172.21.15.2:6822/101416","heartbeat_front_addr":"172.21.15.2:6818/101416","state":["exists","up"]},{"osd":2,"uuid":"817f70a8-c4b5-4470-beca-45061a2bf389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6809","nonce":101415},{"type":"v1","addr":"172.21.15.2:6811","nonce":101415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6813","nonce":101415},{"type":"v1","addr":"172.21.15.2:6815","nonce":101415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6821","nonce":101415},{"type":"v1","addr":"172.21.15.2:6823","nonce":101415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6817","nonce":101415},{"type":"v1","addr":"172.21.15.2:6819","nonce":101415}]},"public_addr":"172.21.15.2:6811/101415","cluster_addr":"172.21.15.2:6815/101415","heartbeat_back_addr":"172.21.15.2:6823/101415","heartbeat_front_addr":"172.21.15.2:6819/101415","state":["exists","up"]},{"osd":3,"uuid":"379bf437-773e-4be5-ac5c-1a8c6b6663e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":101458},{"type":"v1","addr":"172.21.15.2:6825","nonce":101458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":101458},{"type":"v1","addr":"172.21.15.2:6827","nonce":101458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":101458},{"type":"v1","addr":"172.21.15.2:6831","nonce":101458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":101458},{"type":"v1","addr":"172.21.15.2:6829","nonce":101458}]},"public_addr":"172.21.15.2:6825/101458","cluster_addr":"172.21.15.2:6827/101458","heartbeat_back_addr":"172.21.15.2:6831/101458","heartbeat_front_addr":"172.21.15.2:6829/101458","state":["exists","up"]},{"osd":4,"uuid":"7ef04bad-dece-4880-9f9c-4ba20e210e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":101671},{"type":"v1","addr":"172.21.15.85:6809","nonce":101671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":101671},{"type":"v1","addr":"172.21.15.85:6811","nonce":101671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":101671},{"type":"v1","addr":"172.21.15.85:6815","nonce":101671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":101671},{"type":"v1","addr":"172.21.15.85:6813","nonce":101671}]},"public_addr":"172.21.15.85:6809/101671","cluster_addr":"172.21.15.85:6811/101671","heartbeat_back_addr":"172.21.15.85:6815/101671","heartbeat_front_addr":"172.21.15.85:6813/101671","state":["exists","up"]},{"osd":5,"uuid":"9f624f12-fff1-44e7-b4f2-76065630d976","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":101672},{"type":"v1","addr":"172.21.15.85:6801","nonce":101672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":101672},{"type":"v1","addr":"172.21.15.85:6803","nonce":101672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":101672},{"type":"v1","addr":"172.21.15.85:6807","nonce":101672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":101672},{"type":"v1","addr":"172.21.15.85:6805","nonce":101672}]},"public_addr":"172.21.15.85:6801/101672","cluster_addr":"172.21.15.85:6803/101672","heartbeat_back_addr":"172.21.15.85:6807/101672","heartbeat_front_addr":"172.21.15.85:6805/101672","state":["exists","up"]},{"osd":6,"uuid":"ecaf66b0-139a-4a68-8a19-914f382fee4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":101710},{"type":"v1","addr":"172.21.15.85:6825","nonce":101710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":101710},{"type":"v1","addr":"172.21.15.85:6827","nonce":101710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":101710},{"type":"v1","addr":"172.21.15.85:6831","nonce":101710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":101710},{"type":"v1","addr":"172.21.15.85:6829","nonce":101710}]},"public_addr":"172.21.15.85:6825/101710","cluster_addr":"172.21.15.85:6827/101710","heartbeat_back_addr":"172.21.15.85:6831/101710","heartbeat_front_addr":"172.21.15.85:6829/101710","state":["exists","up"]},{"osd":7,"uuid":"e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6816","nonce":101673},{"type":"v1","addr":"172.21.15.85:6817","nonce":101673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":101673},{"type":"v1","addr":"172.21.15.85:6819","nonce":101673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":101673},{"type":"v1","addr":"172.21.15.85:6823","nonce":101673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":101673},{"type":"v1","addr":"172.21.15.85:6821","nonce":101673}]},"public_addr":"172.21.15.85:6817/101673","cluster_addr":"172.21.15.85:6819/101673","heartbeat_back_addr":"172.21.15.85:6823/101673","heartbeat_front_addr":"172.21.15.85:6821/101673","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.388884+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.322448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.322721+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.395855+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.392689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.409320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.377123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.145226+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:40:15.771 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.766+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f3aac066590 msgr2=0x7f3aac068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.771 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.766+0000 7f3ad40af700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f3aac066590 0x7f3aac068a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8000f60 tx=0x7f3ac8007be0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.771 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.766+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0ad460 msgr2=0x7f3acc0c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.772 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.766+0000 7f3ad40af700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0ad460 0x7f3acc0c67d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000d800 tx=0x7f3ac000dbc0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.772 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.767+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 shutdown_connections 2022-12-01T09:40:15.772 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.767+0000 7f3ad40af700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f3aac066590 0x7f3aac068a40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.772 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.767+0000 7f3ad40af700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3acc0c8d70 0x7f3acc15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.773 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.767+0000 7f3ad40af700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3acc070bb0 0x7f3acc0ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.773 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.767+0000 7f3ad40af700 1 --2- 172.21.15.2:0/217917942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3acc0ad460 0x7f3acc0c67d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.773 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.767+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 >> 172.21.15.2:0/217917942 conn(0x7f3acc069aa0 msgr2=0x7f3acc158370 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:15.774 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.767+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 shutdown_connections 2022-12-01T09:40:15.774 INFO:teuthology.misc.health.smithi002.stderr:2022-12-01T09:40:15.767+0000 7f3ad40af700 1 -- 172.21.15.2:0/217917942 wait complete. 2022-12-01T09:40:15.781 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-12-01T09:40:15.782 INFO:tasks.ceph:Creating RBD pool 2022-12-01T09:40:15.782 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd pool create rbd 8 2022-12-01T09:40:15.906 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.903+0000 7f978269b700 1 Processor -- start 2022-12-01T09:40:15.907 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.903+0000 7f978269b700 1 -- start start 2022-12-01T09:40:15.912 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f978269b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f977c15bd30 0x7f977c0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.913 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f978269b700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c0c9c40 con 0x7f977c0ce390 2022-12-01T09:40:15.913 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f978269b700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c15c100 con 0x7f977c06fae0 2022-12-01T09:40:15.913 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f978269b700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c0c9070 con 0x7f977c15bd30 2022-12-01T09:40:15.913 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f977bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f977c15bd30 0x7f977c0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.913 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f9780c38700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f977c0ce390 0x7f977c068e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34076/0 (socket says 172.21.15.2:34076) 2022-12-01T09:40:15.914 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f977bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f977c15bd30 0x7f977c0ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32954/0 (socket says 172.21.15.2:32954) 2022-12-01T09:40:15.914 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f9780c38700 1 -- 172.21.15.2:0/673027679 learned_addr learned my addr 172.21.15.2:0/673027679 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:15.914 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f977bfff700 1 -- 172.21.15.2:0/673027679 >> v1:172.21.15.85:6789/0 conn(0x7f977c06fae0 legacy=0x7f977c0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.915 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f977bfff700 1 -- 172.21.15.2:0/673027679 >> v1:172.21.15.2:6789/0 conn(0x7f977c0ce390 legacy=0x7f977c068e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.915 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f977bfff700 1 -- 172.21.15.2:0/673027679 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c0c9310 con 0x7f977c15bd30 2022-12-01T09:40:15.915 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f977affd700 1 -- 172.21.15.2:0/673027679 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028483595 0 0) 0x7f977c0c9c40 con 0x7f977c0ce390 2022-12-01T09:40:15.915 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.904+0000 7f977bfff700 1 --2- 172.21.15.2:0/673027679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c15bd30 0x7f977c0ca0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9764009840 tx=0x7f9764009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc917a6393e7c97 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:15.916 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f977affd700 1 -- 172.21.15.2:0/673027679 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764011830 con 0x7f977c15bd30 2022-12-01T09:40:15.916 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f977affd700 1 -- 172.21.15.2:0/673027679 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9764011990 con 0x7f977c15bd30 2022-12-01T09:40:15.916 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f977affd700 1 -- 172.21.15.2:0/673027679 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764011d20 con 0x7f977c15bd30 2022-12-01T09:40:15.917 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f978269b700 1 -- 172.21.15.2:0/673027679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c15bd30 msgr2=0x7f977c0ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.917 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f978269b700 1 --2- 172.21.15.2:0/673027679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c15bd30 0x7f977c0ca0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9764009840 tx=0x7f9764009b50 comp rx=0 tx=0).stop 2022-12-01T09:40:15.917 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f978269b700 1 -- 172.21.15.2:0/673027679 shutdown_connections 2022-12-01T09:40:15.917 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f978269b700 1 --2- 172.21.15.2:0/673027679 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c15bd30 0x7f977c0ca0d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.917 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f978269b700 1 -- 172.21.15.2:0/673027679 >> 172.21.15.2:0/673027679 conn(0x7f977c0ce8f0 msgr2=0x7f977c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:15.918 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f978269b700 1 -- 172.21.15.2:0/673027679 shutdown_connections 2022-12-01T09:40:15.918 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.905+0000 7f978269b700 1 -- 172.21.15.2:0/673027679 wait complete. 2022-12-01T09:40:15.918 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.906+0000 7f978269b700 1 Processor -- start 2022-12-01T09:40:15.918 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f978269b700 1 -- start start 2022-12-01T09:40:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f978269b700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f977c06fae0 0x7f977c0c4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f978269b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c0ce390 0x7f977c14d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f978269b700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f977c15bd30 0x7f977c14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.919 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f978269b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0c36b0 con 0x7f977c0ce390 2022-12-01T09:40:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f978269b700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c14ddd0 con 0x7f977c06fae0 2022-12-01T09:40:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f978269b700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c14e0b0 con 0x7f977c15bd30 2022-12-01T09:40:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f977b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c0ce390 0x7f977c14d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.920 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f977b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c0ce390 0x7f977c14d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32958/0 (socket says 172.21.15.2:32958) 2022-12-01T09:40:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.907+0000 7f977b7fe700 1 -- 172.21.15.2:0/3133057931 learned_addr learned my addr 172.21.15.2:0/3133057931 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f977bfff700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f977c06fae0 0x7f977c0c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f9780c38700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f977c15bd30 0x7f977c14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.921 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f977b7fe700 1 -- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f977c15bd30 msgr2=0x7f977c14d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f977b7fe700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f977c15bd30 0x7f977c14d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f977b7fe700 1 -- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f977c06fae0 msgr2=0x7f977c0c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f977b7fe700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f977c06fae0 0x7f977c0c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f977b7fe700 1 -- 172.21.15.2:0/3133057931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97640094a0 con 0x7f977c0ce390 2022-12-01T09:40:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f977b7fe700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c0ce390 0x7f977c14d740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c450 tx=0x7f976c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f9778ff9700 1 -- 172.21.15.2:0/3133057931 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0052d0 con 0x7f977c0ce390 2022-12-01T09:40:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f9778ff9700 1 -- 172.21.15.2:0/3133057931 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c005430 con 0x7f977c0ce390 2022-12-01T09:40:15.923 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c14e390 con 0x7f977c0ce390 2022-12-01T09:40:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f9778ff9700 1 -- 172.21.15.2:0/3133057931 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c0038c0 con 0x7f977c0ce390 2022-12-01T09:40:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.908+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c14e950 con 0x7f977c0ce390 2022-12-01T09:40:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.909+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c068b70 con 0x7f977c0ce390 2022-12-01T09:40:15.924 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.909+0000 7f9778ff9700 1 -- 172.21.15.2:0/3133057931 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f976c019030 con 0x7f977c0ce390 2022-12-01T09:40:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.910+0000 7f9778ff9700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9760066590 0x7f9760068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.910+0000 7f9778ff9700 1 -- 172.21.15.2:0/3133057931 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f976c0824c0 con 0x7f977c0ce390 2022-12-01T09:40:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.910+0000 7f977bfff700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9760066590 0x7f9760068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:15.927 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.910+0000 7f977bfff700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9760066590 0x7f9760068a40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f97640090b0 tx=0x7f976401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:15.927 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:15.914+0000 7f9778ff9700 1 -- 172.21.15.2:0/3133057931 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f976c050080 con 0x7f977c0ce390 2022-12-01T09:40:16.108 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.104+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f977c0c95f0 con 0x7f977c0ce390 2022-12-01T09:40:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.281+0000 7f9778ff9700 1 -- 172.21.15.2:0/3133057931 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7f976c00ce60 con 0x7f977c0ce390 2022-12-01T09:40:16.288 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9760066590 msgr2=0x7f9760068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:16.288 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9760066590 0x7f9760068a40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f97640090b0 tx=0x7f976401c040 comp rx=0 tx=0).stop 2022-12-01T09:40:16.288 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c0ce390 msgr2=0x7f977c14d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c0ce390 0x7f977c14d740 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c450 tx=0x7f976c00c760 comp rx=0 tx=0).stop 2022-12-01T09:40:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 shutdown_connections 2022-12-01T09:40:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9760066590 0x7f9760068a40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f977c15bd30 0x7f977c14d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f977c06fae0 0x7f977c0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 --2- 172.21.15.2:0/3133057931 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f977c0ce390 0x7f977c14d740 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 >> 172.21.15.2:0/3133057931 conn(0x7f977c0ce8f0 msgr2=0x7f977c15abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 shutdown_connections 2022-12-01T09:40:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.284+0000 7f978269b700 1 -- 172.21.15.2:0/3133057931 wait complete. 2022-12-01T09:40:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'rbd' created 2022-12-01T09:40:16.300 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-12-01T09:40:16.429 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.425+0000 7fcb7b609700 1 Processor -- start 2022-12-01T09:40:16.429 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.425+0000 7fcb7b609700 1 -- start start 2022-12-01T09:40:16.435 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.426+0000 7fcb7b609700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fcb741602f0 0x7fcb740c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:16.436 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.426+0000 7fcb7b609700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7406fb40 con 0x7fcb74070900 2022-12-01T09:40:16.436 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.426+0000 7fcb7b609700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7406fe20 con 0x7fcb7415bca0 2022-12-01T09:40:16.436 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.426+0000 7fcb7b609700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb74160860 con 0x7fcb741602f0 2022-12-01T09:40:16.436 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.426+0000 7fcb78ba4700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fcb74070900 0x7fcb740c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34088/0 (socket says 172.21.15.2:34088) 2022-12-01T09:40:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.426+0000 7fcb78ba4700 1 -- 172.21.15.2:0/2547710688 learned_addr learned my addr 172.21.15.2:0/2547710688 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.426+0000 7fcb79ba6700 1 --2- 172.21.15.2:0/2547710688 >> v2:172.21.15.85:3300/0 conn(0x7fcb741602f0 0x7fcb740c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746105215 0 0) 0x7fcb7406fb40 con 0x7fcb74070900 2022-12-01T09:40:16.437 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb5c0035b0 con 0x7fcb74070900 2022-12-01T09:40:16.438 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2145303687 0 0) 0x7fcb5c0035b0 con 0x7fcb74070900 2022-12-01T09:40:16.438 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 >> v2:172.21.15.85:3300/0 conn(0x7fcb741602f0 msgr2=0x7fcb740c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:16.438 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 --2- 172.21.15.2:0/2547710688 >> v2:172.21.15.85:3300/0 conn(0x7fcb741602f0 0x7fcb740c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.438 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 >> v1:172.21.15.85:6789/0 conn(0x7fcb7415bca0 legacy=0x7fcb740cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:16.439 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74070100 con 0x7fcb74070900 2022-12-01T09:40:16.439 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2890946737 0 0) 0x7fcb70001670 con 0x7fcb74070900 2022-12-01T09:40:16.439 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 >> v1:172.21.15.2:6789/0 conn(0x7fcb74070900 legacy=0x7fcb740c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:16.439 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb5c003a00 0x7fcb5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:16.439 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb5c004850 0x7fcb5c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:16.440 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c005200 0x7fcb5c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:16.440 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb74160860 con 0x7fcb5c003a00 2022-12-01T09:40:16.440 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c00c110 con 0x7fcb5c005200 2022-12-01T09:40:16.440 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c00c2d0 con 0x7fcb5c004850 2022-12-01T09:40:16.441 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb793a5700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb5c003a00 0x7fcb5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:16.441 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb79ba6700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c005200 0x7fcb5c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:16.442 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb78ba4700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb5c004850 0x7fcb5c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:16.442 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb793a5700 1 -- 172.21.15.2:0/2547710688 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb5c004850 msgr2=0x7fcb5c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:16.442 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb793a5700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb5c004850 0x7fcb5c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.443 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb793a5700 1 -- 172.21.15.2:0/2547710688 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c005200 msgr2=0x7fcb5c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:16.443 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb793a5700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c005200 0x7fcb5c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.444 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb793a5700 1 -- 172.21.15.2:0/2547710688 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c0035b0 con 0x7fcb5c003a00 2022-12-01T09:40:16.444 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.427+0000 7fcb79ba6700 1 -- 172.21.15.2:0/2547710688 reap_dead start 2022-12-01T09:40:16.445 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb793a5700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb5c003a00 0x7fcb5c003e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb640017c0 tx=0x7fcb5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e32f3722d72b0f server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:16.445 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640052d0 con 0x7fcb5c003a00 2022-12-01T09:40:16.445 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb64005450 con 0x7fcb5c003a00 2022-12-01T09:40:16.446 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb63fff700 1 -- 172.21.15.2:0/2547710688 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640038c0 con 0x7fcb5c003a00 2022-12-01T09:40:16.446 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb7b609700 1 -- 172.21.15.2:0/2547710688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb5c003a00 msgr2=0x7fcb5c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:16.447 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb7b609700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb5c003a00 0x7fcb5c003e10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb640017c0 tx=0x7fcb5c005880 comp rx=0 tx=0).stop 2022-12-01T09:40:16.447 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb7b609700 1 -- 172.21.15.2:0/2547710688 shutdown_connections 2022-12-01T09:40:16.447 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb7b609700 1 --2- 172.21.15.2:0/2547710688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb5c003a00 0x7fcb5c003e10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.448 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb7b609700 1 -- 172.21.15.2:0/2547710688 >> 172.21.15.2:0/2547710688 conn(0x7fcb740ce880 msgr2=0x7fcb740cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:16.448 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb7b609700 1 -- 172.21.15.2:0/2547710688 shutdown_connections 2022-12-01T09:40:16.448 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.428+0000 7fcb7b609700 1 -- 172.21.15.2:0/2547710688 wait complete. 2022-12-01T09:40:16.448 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.429+0000 7fcb7b609700 1 Processor -- start 2022-12-01T09:40:16.449 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb7b609700 1 -- start start 2022-12-01T09:40:16.449 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb7b609700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c003a00 0x7fcb741f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:16.449 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb7b609700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb7415e240 0x7fcb7415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:16.449 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb7b609700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb740ac860 0x7fcb740accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:16.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb7b609700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7415ebf0 con 0x7fcb740ac860 2022-12-01T09:40:16.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb7b609700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb740aea50 con 0x7fcb5c003a00 2022-12-01T09:40:16.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb7b609700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb740aed30 con 0x7fcb7415e240 2022-12-01T09:40:16.450 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb79ba6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb740ac860 0x7fcb740accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:16.451 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb79ba6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb740ac860 0x7fcb740accd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32980/0 (socket says 172.21.15.2:32980) 2022-12-01T09:40:16.451 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.430+0000 7fcb79ba6700 1 -- 172.21.15.2:0/1620832902 learned_addr learned my addr 172.21.15.2:0/1620832902 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:16.451 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb78ba4700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb7415e240 0x7fcb7415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:16.451 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb793a5700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c003a00 0x7fcb741f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:16.451 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb79ba6700 1 -- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb7415e240 msgr2=0x7fcb7415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:16.452 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb79ba6700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb7415e240 0x7fcb7415e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.452 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb79ba6700 1 -- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c003a00 msgr2=0x7fcb741f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:16.452 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb79ba6700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c003a00 0x7fcb741f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:16.452 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb79ba6700 1 -- 172.21.15.2:0/1620832902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb740af010 con 0x7fcb740ac860 2022-12-01T09:40:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb793a5700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c003a00 0x7fcb741f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:40:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb79ba6700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb740ac860 0x7fcb740accd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb740cf960 tx=0x7fcb6c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb61ffb700 1 -- 172.21.15.2:0/1620832902 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00e040 con 0x7fcb740ac860 2022-12-01T09:40:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb61ffb700 1 -- 172.21.15.2:0/1620832902 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c00f040 con 0x7fcb740ac860 2022-12-01T09:40:16.453 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb61ffb700 1 -- 172.21.15.2:0/1620832902 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00fcf0 con 0x7fcb740ac860 2022-12-01T09:40:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb740af2f0 con 0x7fcb740ac860 2022-12-01T09:40:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.431+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb740ad210 con 0x7fcb740ac860 2022-12-01T09:40:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.432+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7404e190 con 0x7fcb740ac860 2022-12-01T09:40:16.454 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.433+0000 7fcb61ffb700 1 -- 172.21.15.2:0/1620832902 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fcb6c022460 con 0x7fcb740ac860 2022-12-01T09:40:16.455 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.433+0000 7fcb61ffb700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fcb54066590 0x7fcb54068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.433+0000 7fcb61ffb700 1 -- 172.21.15.2:0/1620832902 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fcb6c083a60 con 0x7fcb740ac860 2022-12-01T09:40:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.433+0000 7fcb793a5700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fcb54066590 0x7fcb54068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.434+0000 7fcb793a5700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fcb54066590 0x7fcb54068a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb64000c00 tx=0x7fcb64009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:16.456 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.437+0000 7fcb61ffb700 1 -- 172.21.15.2:0/1620832902 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7fcb6c051470 con 0x7fcb740ac860 2022-12-01T09:40:16.633 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:16.629+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcb740703e0 con 0x7fcb740ac860 2022-12-01T09:40:17.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-12-01T09:40:17.273 DEBUG:teuthology.orchestra.run.smithi085:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-12-01T09:40:17.295 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.291+0000 7fcb61ffb700 1 -- 172.21.15.2:0/1620832902 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7fcb6c054a90 con 0x7fcb740ac860 2022-12-01T09:40:17.299 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.295+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fcb54066590 msgr2=0x7fcb54068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.299 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.295+0000 7fcb7b609700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fcb54066590 0x7fcb54068a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb64000c00 tx=0x7fcb64009030 comp rx=0 tx=0).stop 2022-12-01T09:40:17.300 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.295+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb740ac860 msgr2=0x7fcb740accd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.300 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.295+0000 7fcb7b609700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb740ac860 0x7fcb740accd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb740cf960 tx=0x7fcb6c011f80 comp rx=0 tx=0).stop 2022-12-01T09:40:17.301 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.298+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 shutdown_connections 2022-12-01T09:40:17.301 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.298+0000 7fcb7b609700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fcb54066590 0x7fcb54068a40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.301 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.298+0000 7fcb7b609700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb740ac860 0x7fcb740accd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.302 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.298+0000 7fcb7b609700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fcb5c003a00 0x7fcb741f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.302 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.298+0000 7fcb7b609700 1 --2- 172.21.15.2:0/1620832902 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fcb7415e240 0x7fcb7415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.302 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.298+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 >> 172.21.15.2:0/1620832902 conn(0x7fcb740ce880 msgr2=0x7fcb7406cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:17.302 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.298+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 shutdown_connections 2022-12-01T09:40:17.303 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.298+0000 7fcb7b609700 1 -- 172.21.15.2:0/1620832902 wait complete. 2022-12-01T09:40:17.303 INFO:teuthology.orchestra.run.smithi002.stderr:enabled application 'rbd' on pool 'rbd' 2022-12-01T09:40:17.317 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-12-01T09:40:17.318 INFO:tasks.ceph.mds.a:Restarting daemon 2022-12-01T09:40:17.318 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-12-01T09:40:17.379 INFO:tasks.ceph.mds.a:Started 2022-12-01T09:40:17.380 INFO:tasks.ceph.mds.c:Restarting daemon 2022-12-01T09:40:17.380 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-12-01T09:40:17.383 INFO:tasks.ceph.mds.c:Started 2022-12-01T09:40:17.383 INFO:tasks.ceph.mds.b:Restarting daemon 2022-12-01T09:40:17.384 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-12-01T09:40:17.387 INFO:tasks.ceph.mds.b:Started 2022-12-01T09:40:17.387 INFO:tasks.ceph.mds.d:Restarting daemon 2022-12-01T09:40:17.387 DEBUG:teuthology.orchestra.run.smithi085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-12-01T09:40:17.390 INFO:tasks.ceph.mds.d:Started 2022-12-01T09:40:17.390 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-12-01T09:40:17.391 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-12-01T09:40:17.468 INFO:tasks.ceph.mds.a.smithi002.stdout:starting mds.a at 2022-12-01T09:40:17.469 INFO:tasks.ceph.mds.c.smithi002.stdout:starting mds.c at 2022-12-01T09:40:17.470 INFO:tasks.ceph.mds.d.smithi085.stdout:starting mds.d at 2022-12-01T09:40:17.470 INFO:tasks.ceph.mds.b.smithi085.stdout:starting mds.b at 2022-12-01T09:40:17.528 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.525+0000 7f69bbefd700 1 Processor -- start 2022-12-01T09:40:17.529 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.525+0000 7f69bbefd700 1 -- start start 2022-12-01T09:40:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69bbefd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b40ca0c0 0x7f69b40cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69bbefd700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b40ce740 con 0x7f69b406fae0 2022-12-01T09:40:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69bbefd700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b406bf80 con 0x7f69b406d3f0 2022-12-01T09:40:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69bbefd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40c9650 con 0x7f69b40ca0c0 2022-12-01T09:40:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69ba49a700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f69b406fae0 0x7f69b406c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34134/0 (socket says 172.21.15.2:34134) 2022-12-01T09:40:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69ba49a700 1 -- 172.21.15.2:0/1519796653 learned_addr learned my addr 172.21.15.2:0/1519796653 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69b9c99700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b40ca0c0 0x7f69b40cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127828081 0 0) 0x7f69b40ce740 con 0x7f69b406fae0 2022-12-01T09:40:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.526+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f699c0035b0 con 0x7f69b406fae0 2022-12-01T09:40:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333242913 0 0) 0x7f69b406bf80 con 0x7f69b406d3f0 2022-12-01T09:40:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 --> v1:172.21.15.85:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69b40ce740 con 0x7f69b406d3f0 2022-12-01T09:40:17.539 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3895581106 0 0) 0x7f699c0035b0 con 0x7f69b406fae0 2022-12-01T09:40:17.539 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b40ca0c0 msgr2=0x7f69b40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:17.539 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b40ca0c0 0x7f69b40cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.539 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 >> v1:172.21.15.85:6789/0 conn(0x7f69b406d3f0 legacy=0x7f69b40c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.540 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40ce180 con 0x7f69b406fae0 2022-12-01T09:40:17.540 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2890946737 0 0) 0x7f69b0001670 con 0x7f69b406fae0 2022-12-01T09:40:17.540 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 >> v1:172.21.15.2:6789/0 conn(0x7f69b406fae0 legacy=0x7f69b406c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.540 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f699c003c30 0x7f699c004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f699c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b40ca0c0 0x7f699c0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40c9650 con 0x7f699c004850 2022-12-01T09:40:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c00bbd0 con 0x7f699c003c30 2022-12-01T09:40:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c00bd90 con 0x7f69b40ca0c0 2022-12-01T09:40:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b9498700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f699c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69ba49a700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b40ca0c0 0x7f699c0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b9c99700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f699c003c30 0x7f699c004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b9498700 1 -- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b40ca0c0 msgr2=0x7f699c0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:17.544 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b9498700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b40ca0c0 0x7f699c0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.544 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b9498700 1 -- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f699c003c30 msgr2=0x7f699c004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:17.544 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b9498700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f699c003c30 0x7f699c004020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.544 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b9498700 1 -- 172.21.15.2:0/1519796653 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0035b0 con 0x7f699c004850 2022-12-01T09:40:17.545 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.527+0000 7f69b9498700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f699c004cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007fd0 tx=0x7f699c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162348b2f466f51a server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:17.545 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4001430 con 0x7f699c004850 2022-12-01T09:40:17.545 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a40052d0 con 0x7f699c004850 2022-12-01T09:40:17.545 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69b8c97700 1 -- 172.21.15.2:0/1519796653 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4001d40 con 0x7f699c004850 2022-12-01T09:40:17.546 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69bbefd700 1 -- 172.21.15.2:0/1519796653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 msgr2=0x7f699c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.546 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69bbefd700 1 --2- 172.21.15.2:0/1519796653 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f699c004cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f69a4007fd0 tx=0x7f699c005880 comp rx=0 tx=0).stop 2022-12-01T09:40:17.546 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69ba49a700 1 -- 172.21.15.2:0/1519796653 reap_dead start 2022-12-01T09:40:17.546 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69bbefd700 1 -- 172.21.15.2:0/1519796653 shutdown_connections 2022-12-01T09:40:17.546 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69bbefd700 1 -- 172.21.15.2:0/1519796653 >> 172.21.15.2:0/1519796653 conn(0x7f69b40ce8f0 msgr2=0x7f69b40cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:17.547 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69bbefd700 1 -- 172.21.15.2:0/1519796653 shutdown_connections 2022-12-01T09:40:17.547 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.528+0000 7f69bbefd700 1 -- 172.21.15.2:0/1519796653 wait complete. 2022-12-01T09:40:17.547 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.529+0000 7f69bbefd700 1 Processor -- start 2022-12-01T09:40:17.547 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.529+0000 7f69bbefd700 1 -- start start 2022-12-01T09:40:17.548 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69bbefd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f69b41f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.548 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69bbefd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f69b40c2c80 0x7f69b414dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.548 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69bbefd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b415c8e0 0x7f69b415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.548 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69bbefd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b415d2c0 con 0x7f699c004850 2022-12-01T09:40:17.549 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69bbefd700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40b2a70 con 0x7f69b40c2c80 2022-12-01T09:40:17.549 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69bbefd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40c30f0 con 0x7f69b415c8e0 2022-12-01T09:40:17.549 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9c99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f69b41f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.549 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9c99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f69b41f1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33020/0 (socket says 172.21.15.2:33020) 2022-12-01T09:40:17.550 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9c99700 1 -- 172.21.15.2:0/4121877514 learned_addr learned my addr 172.21.15.2:0/4121877514 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:17.550 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9498700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f69b40c2c80 0x7f69b414dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.550 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69ba49a700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b415c8e0 0x7f69b415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.550 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9c99700 1 -- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b415c8e0 msgr2=0x7f69b415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.551 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9c99700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b415c8e0 0x7f69b415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.551 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9c99700 1 -- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f69b40c2c80 msgr2=0x7f69b414dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.551 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9c99700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f69b40c2c80 0x7f69b414dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.551 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9c99700 1 -- 172.21.15.2:0/4121877514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b41f2580 con 0x7f699c004850 2022-12-01T09:40:17.552 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.530+0000 7f69b9498700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f69b40c2c80 0x7f69b414dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:40:17.552 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.531+0000 7f69b9c99700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f69b41f1f90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009f90 tx=0x7f69ac00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:17.552 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.531+0000 7f69a27fc700 1 -- 172.21.15.2:0/4121877514 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00e040 con 0x7f699c004850 2022-12-01T09:40:17.552 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.531+0000 7f69a27fc700 1 -- 172.21.15.2:0/4121877514 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac00f040 con 0x7f699c004850 2022-12-01T09:40:17.553 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.531+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b41f2860 con 0x7f699c004850 2022-12-01T09:40:17.553 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.531+0000 7f69a27fc700 1 -- 172.21.15.2:0/4121877514 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac015650 con 0x7f699c004850 2022-12-01T09:40:17.553 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.531+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b41f2e20 con 0x7f699c004850 2022-12-01T09:40:17.553 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.532+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b4068b70 con 0x7f699c004850 2022-12-01T09:40:17.553 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.532+0000 7f69a27fc700 1 -- 172.21.15.2:0/4121877514 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f69ac0157b0 con 0x7f699c004850 2022-12-01T09:40:17.554 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.533+0000 7f69a27fc700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f6994066590 0x7f6994068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.554 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.533+0000 7f69a27fc700 1 -- 172.21.15.2:0/4121877514 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f69ac084140 con 0x7f699c004850 2022-12-01T09:40:17.554 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.533+0000 7f69b9498700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f6994066590 0x7f6994068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.555 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.533+0000 7f69b9498700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f6994066590 0x7f6994068a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69b41efbb0 tx=0x7f69a400a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:17.555 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.537+0000 7f69a27fc700 1 -- 172.21.15.2:0/4121877514 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f69ac051b20 con 0x7f699c004850 2022-12-01T09:40:17.735 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.731+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69b40ce460 con 0x7f699c004850 2022-12-01T09:40:17.736 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.732+0000 7f69a27fc700 1 -- 172.21.15.2:0/4121877514 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13658 (secure 0 0 0) 0x7f69ac055140 con 0x7f699c004850 2022-12-01T09:40:17.736 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:40:17.736 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":19,"fsid":"3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803","created":"2022-12-01T09:39:56.873720+0000","modified":"2022-12-01T09:40:17.281367+0000","last_up_change":"2022-12-01T09:40:08.916618+0000","last_in_change":"2022-12-01T09:40:00.145489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-12-01T09:40:08.277187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-12-01T09:40:16.135544+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"29ad9800-3b0e-4728-9c4b-27eed3978c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":101414},{"type":"v1","addr":"172.21.15.2:6801","nonce":101414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":101414},{"type":"v1","addr":"172.21.15.2:6803","nonce":101414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":101414},{"type":"v1","addr":"172.21.15.2:6807","nonce":101414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":101414},{"type":"v1","addr":"172.21.15.2:6805","nonce":101414}]},"public_addr":"172.21.15.2:6801/101414","cluster_addr":"172.21.15.2:6803/101414","heartbeat_back_addr":"172.21.15.2:6807/101414","heartbeat_front_addr":"172.21.15.2:6805/101414","state":["exists","up"]},{"osd":1,"uuid":"51463426-88ae-4c28-880b-1b655faff15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":101416},{"type":"v1","addr":"172.21.15.2:6810","nonce":101416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":101416},{"type":"v1","addr":"172.21.15.2:6814","nonce":101416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":101416},{"type":"v1","addr":"172.21.15.2:6822","nonce":101416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":101416},{"type":"v1","addr":"172.21.15.2:6818","nonce":101416}]},"public_addr":"172.21.15.2:6810/101416","cluster_addr":"172.21.15.2:6814/101416","heartbeat_back_addr":"172.21.15.2:6822/101416","heartbeat_front_addr":"172.21.15.2:6818/101416","state":["exists","up"]},{"osd":2,"uuid":"817f70a8-c4b5-4470-beca-45061a2bf389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6809","nonce":101415},{"type":"v1","addr":"172.21.15.2:6811","nonce":101415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6813","nonce":101415},{"type":"v1","addr":"172.21.15.2:6815","nonce":101415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6821","nonce":101415},{"type":"v1","addr":"172.21.15.2:6823","nonce":101415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6817","nonce":101415},{"type":"v1","addr":"172.21.15.2:6819","nonce":101415}]},"public_addr":"172.21.15.2:6811/101415","cluster_addr":"172.21.15.2:6815/101415","heartbeat_back_addr":"172.21.15.2:6823/101415","heartbeat_front_addr":"172.21.15.2:6819/101415","state":["exists","up"]},{"osd":3,"uuid":"379bf437-773e-4be5-ac5c-1a8c6b6663e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":101458},{"type":"v1","addr":"172.21.15.2:6825","nonce":101458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":101458},{"type":"v1","addr":"172.21.15.2:6827","nonce":101458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":101458},{"type":"v1","addr":"172.21.15.2:6831","nonce":101458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":101458},{"type":"v1","addr":"172.21.15.2:6829","nonce":101458}]},"public_addr":"172.21.15.2:6825/101458","cluster_addr":"172.21.15.2:6827/101458","heartbeat_back_addr":"172.21.15.2:6831/101458","heartbeat_front_addr":"172.21.15.2:6829/101458","state":["exists","up"]},{"osd":4,"uuid":"7ef04bad-dece-4880-9f9c-4ba20e210e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":101671},{"type":"v1","addr":"172.21.15.85:6809","nonce":101671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":101671},{"type":"v1","addr":"172.21.15.85:6811","nonce":101671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":101671},{"type":"v1","addr":"172.21.15.85:6815","nonce":101671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":101671},{"type":"v1","addr":"172.21.15.85:6813","nonce":101671}]},"public_addr":"172.21.15.85:6809/101671","cluster_addr":"172.21.15.85:6811/101671","heartbeat_back_addr":"172.21.15.85:6815/101671","heartbeat_front_addr":"172.21.15.85:6813/101671","state":["exists","up"]},{"osd":5,"uuid":"9f624f12-fff1-44e7-b4f2-76065630d976","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":101672},{"type":"v1","addr":"172.21.15.85:6801","nonce":101672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":101672},{"type":"v1","addr":"172.21.15.85:6803","nonce":101672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":101672},{"type":"v1","addr":"172.21.15.85:6807","nonce":101672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":101672},{"type":"v1","addr":"172.21.15.85:6805","nonce":101672}]},"public_addr":"172.21.15.85:6801/101672","cluster_addr":"172.21.15.85:6803/101672","heartbeat_back_addr":"172.21.15.85:6807/101672","heartbeat_front_addr":"172.21.15.85:6805/101672","state":["exists","up"]},{"osd":6,"uuid":"ecaf66b0-139a-4a68-8a19-914f382fee4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":101710},{"type":"v1","addr":"172.21.15.85:6825","nonce":101710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":101710},{"type":"v1","addr":"172.21.15.85:6827","nonce":101710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":101710},{"type":"v1","addr":"172.21.15.85:6831","nonce":101710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":101710},{"type":"v1","addr":"172.21.15.85:6829","nonce":101710}]},"public_addr":"172.21.15.85:6825/101710","cluster_addr":"172.21.15.85:6827/101710","heartbeat_back_addr":"172.21.15.85:6831/101710","heartbeat_front_addr":"172.21.15.85:6829/101710","state":["exists","up"]},{"osd":7,"uuid":"e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6816","nonce":101673},{"type":"v1","addr":"172.21.15.85:6817","nonce":101673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":101673},{"type":"v1","addr":"172.21.15.85:6819","nonce":101673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":101673},{"type":"v1","addr":"172.21.15.85:6823","nonce":101673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":101673},{"type":"v1","addr":"172.21.15.85:6821","nonce":101673}]},"public_addr":"172.21.15.85:6817/101673","cluster_addr":"172.21.15.85:6819/101673","heartbeat_back_addr":"172.21.15.85:6823/101673","heartbeat_front_addr":"172.21.15.85:6821/101673","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.388884+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.322448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.322721+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.395855+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.392689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.409320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.377123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.145226+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:40:17.738 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f6994066590 msgr2=0x7f6994068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.738 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f6994066590 0x7f6994068a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69b41efbb0 tx=0x7f69a400a000 comp rx=0 tx=0).stop 2022-12-01T09:40:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 msgr2=0x7f69b41f1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f69b41f1f90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009f90 tx=0x7f69ac00f490 comp rx=0 tx=0).stop 2022-12-01T09:40:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 shutdown_connections 2022-12-01T09:40:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f6994066590 0x7f6994068a40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f69b415c8e0 0x7f69b415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f699c004850 0x7f69b41f1f90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 --2- 172.21.15.2:0/4121877514 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f69b40c2c80 0x7f69b414dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.740 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.734+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 >> 172.21.15.2:0/4121877514 conn(0x7f69b40ce8f0 msgr2=0x7f69b40ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:17.741 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.735+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 shutdown_connections 2022-12-01T09:40:17.741 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.735+0000 7f69bbefd700 1 -- 172.21.15.2:0/4121877514 wait complete. 2022-12-01T09:40:17.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-12-01T09:40:08.277187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-12-01T09:40:16.135544+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-12-01T09:40:17.751 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-12-01T09:40:17.879 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.875+0000 7f4bc5c2d700 1 Processor -- start 2022-12-01T09:40:17.879 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.875+0000 7f4bc5c2d700 1 -- start start 2022-12-01T09:40:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bc5c2d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4bc00cd890 0x7f4bc00c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bc5c2d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc00c9880 con 0x7f4bc00cb760 2022-12-01T09:40:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bc5c2d700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc015d690 con 0x7f4bc00b12c0 2022-12-01T09:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bc5c2d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc015cb10 con 0x7f4bc00cd890 2022-12-01T09:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbffff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4bc00cb760 0x7f4bc00ca230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34154/0 (socket says 172.21.15.2:34154) 2022-12-01T09:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbeffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4bc00cd890 0x7f4bc00c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbffff700 1 -- 172.21.15.2:0/3111828522 learned_addr learned my addr 172.21.15.2:0/3111828522 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbeffd700 1 -- 172.21.15.2:0/3111828522 >> v1:172.21.15.85:6789/0 conn(0x7f4bc00b12c0 legacy=0x7f4bc00cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbeffd700 1 -- 172.21.15.2:0/3111828522 >> v1:172.21.15.2:6789/0 conn(0x7f4bc00cb760 legacy=0x7f4bc00ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbeffd700 1 -- 172.21.15.2:0/3111828522 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc015cdf0 con 0x7f4bc00cd890 2022-12-01T09:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbe7fc700 1 -- 172.21.15.2:0/3111828522 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242531722 0 0) 0x7f4bc015d690 con 0x7f4bc00b12c0 2022-12-01T09:40:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbeffd700 1 --2- 172.21.15.2:0/3111828522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc00c9cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009840 tx=0x7f4ba8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8366d45b9bcc02 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbe7fc700 1 -- 172.21.15.2:0/3111828522 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8004230 con 0x7f4bc00cd890 2022-12-01T09:40:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbe7fc700 1 -- 172.21.15.2:0/3111828522 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba8004390 con 0x7f4bc00cd890 2022-12-01T09:40:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.877+0000 7f4bbe7fc700 1 -- 172.21.15.2:0/3111828522 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80036a0 con 0x7f4bc00cd890 2022-12-01T09:40:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.878+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3111828522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 msgr2=0x7f4bc00c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.878+0000 7f4bc5c2d700 1 --2- 172.21.15.2:0/3111828522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc00c9cf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009840 tx=0x7f4ba8009b50 comp rx=0 tx=0).stop 2022-12-01T09:40:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.878+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3111828522 shutdown_connections 2022-12-01T09:40:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.878+0000 7f4bc5c2d700 1 --2- 172.21.15.2:0/3111828522 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc00c9cf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.878+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3111828522 >> 172.21.15.2:0/3111828522 conn(0x7f4bc00cbc00 msgr2=0x7f4bc00cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.878+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3111828522 shutdown_connections 2022-12-01T09:40:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.878+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3111828522 wait complete. 2022-12-01T09:40:17.890 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.879+0000 7f4bc5c2d700 1 Processor -- start 2022-12-01T09:40:17.891 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.880+0000 7f4bc5c2d700 1 -- start start 2022-12-01T09:40:17.891 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bc5c2d700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4bc00b12c0 0x7f4bc00ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.891 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bc5c2d700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4bc00cb760 0x7f4bc00c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.892 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bc5c2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc0150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.892 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bc5c2d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00c4a50 con 0x7f4bc00cd890 2022-12-01T09:40:17.892 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bc5c2d700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00c3ed0 con 0x7f4bc00b12c0 2022-12-01T09:40:17.892 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bc5c2d700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00acce0 con 0x7f4bc00cb760 2022-12-01T09:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc0150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc0150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33038/0 (socket says 172.21.15.2:33038) 2022-12-01T09:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbffff700 1 -- 172.21.15.2:0/3682264620 learned_addr learned my addr 172.21.15.2:0/3682264620 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:17.893 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbeffd700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4bc00cb760 0x7f4bc00c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.894 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbf7fe700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4bc00b12c0 0x7f4bc00ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.894 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbeffd700 1 -- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4bc00b12c0 msgr2=0x7f4bc00ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.894 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbeffd700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4bc00b12c0 0x7f4bc00ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.894 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbeffd700 1 -- 172.21.15.2:0/3682264620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 msgr2=0x7f4bc0150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:17.895 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbeffd700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc0150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:17.895 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbeffd700 1 -- 172.21.15.2:0/3682264620 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80094a0 con 0x7f4bc00cb760 2022-12-01T09:40:17.895 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbf7fe700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4bc00b12c0 0x7f4bc00ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:40:17.895 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbffff700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc0150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:40:17.896 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bbeffd700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4bc00cb760 0x7f4bc00c4ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0071ab0 tx=0x7f4ba8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:17.896 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.881+0000 7f4bb7fff700 1 -- 172.21.15.2:0/3682264620 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8011630 con 0x7f4bc00cb760 2022-12-01T09:40:17.896 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.882+0000 7f4bb7fff700 1 -- 172.21.15.2:0/3682264620 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba801f070 con 0x7f4bc00cb760 2022-12-01T09:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.882+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc00c4120 con 0x7f4bc00cb760 2022-12-01T09:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.882+0000 7f4bb7fff700 1 -- 172.21.15.2:0/3682264620 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80119f0 con 0x7f4bc00cb760 2022-12-01T09:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.882+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc00c46e0 con 0x7f4bc00cb760 2022-12-01T09:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.882+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0068b70 con 0x7f4bc00cb760 2022-12-01T09:40:17.897 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.883+0000 7f4bb7fff700 1 -- 172.21.15.2:0/3682264620 <== mon.2 v2:172.21.15.85:3301/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f4ba8017020 con 0x7f4bc00cb760 2022-12-01T09:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.883+0000 7f4bb7fff700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f4ba4066590 0x7f4ba4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.883+0000 7f4bb7fff700 1 -- 172.21.15.2:0/3682264620 <== mon.2 v2:172.21.15.85:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4ba8021070 con 0x7f4bc00cb760 2022-12-01T09:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.883+0000 7f4bbf7fe700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f4ba4066590 0x7f4ba4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:17.898 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.884+0000 7f4bbf7fe700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f4ba4066590 0x7f4ba4068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0001120 tx=0x7f4bb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:17.899 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:17.888+0000 7f4bb7fff700 1 -- 172.21.15.2:0/3682264620 <== mon.2 v2:172.21.15.85:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f4ba8089070 con 0x7f4bc00cb760 2022-12-01T09:40:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.077+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bc00adc50 con 0x7f4bc00cb760 2022-12-01T09:40:18.082 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.078+0000 7f4bb7fff700 1 -- 172.21.15.2:0/3682264620 <== mon.2 v2:172.21.15.85:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ba802b020 con 0x7f4bc00cb760 2022-12-01T09:40:18.084 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2022-12-01T09:40:18.086 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f4ba4066590 msgr2=0x7f4ba4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.087 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f4ba4066590 0x7f4ba4068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0001120 tx=0x7f4bb0009040 comp rx=0 tx=0).stop 2022-12-01T09:40:18.087 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4bc00cb760 msgr2=0x7f4bc00c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.088 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4bc00cb760 0x7f4bc00c4ee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0071ab0 tx=0x7f4ba8004750 comp rx=0 tx=0).stop 2022-12-01T09:40:18.088 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 shutdown_connections 2022-12-01T09:40:18.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f4ba4066590 0x7f4ba4068a40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4bc00cd890 0x7f4bc0150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.089 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4bc00b12c0 0x7f4bc00ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.090 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 --2- 172.21.15.2:0/3682264620 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4bc00cb760 0x7f4bc00c4ee0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.090 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 >> 172.21.15.2:0/3682264620 conn(0x7f4bc00cbc00 msgr2=0x7f4bc0070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:18.091 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 shutdown_connections 2022-12-01T09:40:18.091 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.080+0000 7f4bc5c2d700 1 -- 172.21.15.2:0/3682264620 wait complete. 2022-12-01T09:40:18.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-12-01T09:40:18.223 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.219+0000 7f184abd3700 1 Processor -- start 2022-12-01T09:40:18.223 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.219+0000 7f184abd3700 1 -- start start 2022-12-01T09:40:18.226 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.220+0000 7f184abd3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184415bd30 0x7f18440ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.226 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.220+0000 7f184abd3700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f18440ce390 0x7f184406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.227 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.220+0000 7f184abd3700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440c9c40 con 0x7f184406fae0 2022-12-01T09:40:18.227 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.220+0000 7f184abd3700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1844070490 con 0x7f18440ce390 2022-12-01T09:40:18.227 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.220+0000 7f184abd3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184415c100 con 0x7f184415bd30 2022-12-01T09:40:18.227 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.220+0000 7f1849170700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f18440ce390 0x7f184406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.227 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.220+0000 7f1843fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f184406fae0 0x7f18440cdc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34164/0 (socket says 172.21.15.2:34164) 2022-12-01T09:40:18.228 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.220+0000 7f1843fff700 1 -- 172.21.15.2:0/2056103693 learned_addr learned my addr 172.21.15.2:0/2056103693 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:18.228 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f184896f700 1 --2- 172.21.15.2:0/2056103693 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184415bd30 0x7f18440ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.228 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f1849170700 1 -- 172.21.15.2:0/2056103693 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184415bd30 msgr2=0x7f18440ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:18.228 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f1849170700 1 --2- 172.21.15.2:0/2056103693 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184415bd30 0x7f18440ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.229 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f1849170700 1 -- 172.21.15.2:0/2056103693 >> v1:172.21.15.2:6789/0 conn(0x7f184406fae0 legacy=0x7f18440cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.229 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f1849170700 1 -- 172.21.15.2:0/2056103693 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440c9070 con 0x7f18440ce390 2022-12-01T09:40:18.229 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f18437fe700 1 -- 172.21.15.2:0/2056103693 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204237489 0 0) 0x7f18440c9c40 con 0x7f184406fae0 2022-12-01T09:40:18.229 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f1849170700 1 --2- 172.21.15.2:0/2056103693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18440ce390 0x7f184406ff50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f183c009d90 tx=0x7f183c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c31cf4f237d00a6 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.229 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f18437fe700 1 -- 172.21.15.2:0/2056103693 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c00e070 con 0x7f18440ce390 2022-12-01T09:40:18.230 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f18437fe700 1 -- 172.21.15.2:0/2056103693 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c00bb20 con 0x7f18440ce390 2022-12-01T09:40:18.230 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.221+0000 7f18437fe700 1 -- 172.21.15.2:0/2056103693 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c012430 con 0x7f18440ce390 2022-12-01T09:40:18.230 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.222+0000 7f184abd3700 1 -- 172.21.15.2:0/2056103693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18440ce390 msgr2=0x7f184406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.230 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.222+0000 7f184abd3700 1 --2- 172.21.15.2:0/2056103693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18440ce390 0x7f184406ff50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f183c009d90 tx=0x7f183c00b330 comp rx=0 tx=0).stop 2022-12-01T09:40:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.222+0000 7f184abd3700 1 -- 172.21.15.2:0/2056103693 shutdown_connections 2022-12-01T09:40:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.222+0000 7f184abd3700 1 --2- 172.21.15.2:0/2056103693 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18440ce390 0x7f184406ff50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.222+0000 7f184abd3700 1 --2- 172.21.15.2:0/2056103693 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184415bd30 0x7f18440ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.222+0000 7f184abd3700 1 -- 172.21.15.2:0/2056103693 >> 172.21.15.2:0/2056103693 conn(0x7f18440ce8f0 msgr2=0x7f18440cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:18.231 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.222+0000 7f184abd3700 1 -- 172.21.15.2:0/2056103693 shutdown_connections 2022-12-01T09:40:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.222+0000 7f184abd3700 1 -- 172.21.15.2:0/2056103693 wait complete. 2022-12-01T09:40:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.223+0000 7f184abd3700 1 Processor -- start 2022-12-01T09:40:18.232 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.223+0000 7f184abd3700 1 -- start start 2022-12-01T09:40:18.235 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f184abd3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184406fae0 0x7f18441f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.236 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f184abd3700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18440ce390 0x7f18440ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.236 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f184abd3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f184415bd30 0x7f184414aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.236 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f184abd3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184414b4e0 con 0x7f184415bd30 2022-12-01T09:40:18.236 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f184abd3700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184414b7c0 con 0x7f18440ce390 2022-12-01T09:40:18.237 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f184abd3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18441f4900 con 0x7f184406fae0 2022-12-01T09:40:18.237 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f184415bd30 0x7f184414aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f184415bd30 0x7f184414aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33052/0 (socket says 172.21.15.2:33052) 2022-12-01T09:40:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 -- 172.21.15.2:0/2467129546 learned_addr learned my addr 172.21.15.2:0/2467129546 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1843fff700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18440ce390 0x7f18440ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f184896f700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184406fae0 0x7f18441f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 -- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184406fae0 msgr2=0x7f18441f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184406fae0 0x7f18441f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 -- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18440ce390 msgr2=0x7f18440ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.239 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18440ce390 0x7f18440ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.240 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 -- 172.21.15.2:0/2467129546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c009a70 con 0x7f184415bd30 2022-12-01T09:40:18.240 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.224+0000 7f1849170700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f184415bd30 0x7f184414aa90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18440cf420 tx=0x7f183c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.240 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.225+0000 7f18417fa700 1 -- 172.21.15.2:0/2467129546 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c012430 con 0x7f184415bd30 2022-12-01T09:40:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.225+0000 7f18417fa700 1 -- 172.21.15.2:0/2467129546 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c020070 con 0x7f184415bd30 2022-12-01T09:40:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.225+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18441f4be0 con 0x7f184415bd30 2022-12-01T09:40:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.225+0000 7f18417fa700 1 -- 172.21.15.2:0/2467129546 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c004280 con 0x7f184415bd30 2022-12-01T09:40:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.225+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184414ec80 con 0x7f184415bd30 2022-12-01T09:40:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.226+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18441f4e20 con 0x7f184415bd30 2022-12-01T09:40:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.227+0000 7f18417fa700 1 -- 172.21.15.2:0/2467129546 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f183c02e070 con 0x7f184415bd30 2022-12-01T09:40:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.227+0000 7f18417fa700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f182c066590 0x7f182c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.228+0000 7f18417fa700 1 -- 172.21.15.2:0/2467129546 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f183c022070 con 0x7f184415bd30 2022-12-01T09:40:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.228+0000 7f184896f700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f182c066590 0x7f182c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.228+0000 7f184896f700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f182c066590 0x7f182c068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1838009be0 tx=0x7f1838009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.231+0000 7f18417fa700 1 -- 172.21.15.2:0/2467129546 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f183c052fc0 con 0x7f184415bd30 2022-12-01T09:40:18.426 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.423+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f18441f4dd0 con 0x7f184415bd30 2022-12-01T09:40:18.426 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.423+0000 7f18417fa700 1 -- 172.21.15.2:0/2467129546 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f183c02c0a0 con 0x7f184415bd30 2022-12-01T09:40:18.427 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2022-12-01T09:40:18.429 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f182c066590 msgr2=0x7f182c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.430 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f182c066590 0x7f182c068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1838009be0 tx=0x7f1838009210 comp rx=0 tx=0).stop 2022-12-01T09:40:18.430 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f184415bd30 msgr2=0x7f184414aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.431 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f184415bd30 0x7f184414aa90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18440cf420 tx=0x7f183c004ad0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.431 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 shutdown_connections 2022-12-01T09:40:18.432 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f182c066590 0x7f182c068a40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.432 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f184415bd30 0x7f184414aa90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.433 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f184406fae0 0x7f18441f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.433 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 --2- 172.21.15.2:0/2467129546 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f18440ce390 0x7f18440ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.434 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 >> 172.21.15.2:0/2467129546 conn(0x7f18440ce8f0 msgr2=0x7f1844069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:18.434 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 shutdown_connections 2022-12-01T09:40:18.434 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.425+0000 7f184abd3700 1 -- 172.21.15.2:0/2467129546 wait complete. 2022-12-01T09:40:18.440 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-12-01T09:40:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.564+0000 7f9c6a010700 1 Processor -- start 2022-12-01T09:40:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.564+0000 7f9c6a010700 1 -- start start 2022-12-01T09:40:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c6a010700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f9c6406dad0 0x7f9c64068e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c6a010700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6406cd40 con 0x7f9c6406c940 2022-12-01T09:40:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c6a010700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c640cde20 con 0x7f9c640693a0 2022-12-01T09:40:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c6a010700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c64069840 con 0x7f9c6406dad0 2022-12-01T09:40:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c63fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f9c6406c940 0x7f9c6415f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34186/0 (socket says 172.21.15.2:34186) 2022-12-01T09:40:18.572 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c63fff700 1 -- 172.21.15.2:0/2346144345 learned_addr learned my addr 172.21.15.2:0/2346144345 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:18.572 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c637fe700 1 --2- 172.21.15.2:0/2346144345 >> v2:172.21.15.85:3300/0 conn(0x7f9c6406dad0 0x7f9c64068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.572 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c627fc700 1 -- 172.21.15.2:0/2346144345 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232428543 0 0) 0x7f9c5c001490 con 0x7f9c6406c940 2022-12-01T09:40:18.572 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c627fc700 1 -- 172.21.15.2:0/2346144345 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c440035b0 con 0x7f9c6406c940 2022-12-01T09:40:18.573 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c637fe700 1 -- 172.21.15.2:0/2346144345 >> v1:172.21.15.85:6789/0 conn(0x7f9c640693a0 legacy=0x7f9c6406c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.573 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c627fc700 1 -- 172.21.15.2:0/2346144345 <== mon.1 v1:172.21.15.85:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272391604 0 0) 0x7f9c640cde20 con 0x7f9c640693a0 2022-12-01T09:40:18.573 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c637fe700 1 -- 172.21.15.2:0/2346144345 >> v1:172.21.15.2:6789/0 conn(0x7f9c6406c940 legacy=0x7f9c6415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.573 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.565+0000 7f9c637fe700 1 -- 172.21.15.2:0/2346144345 --> v2:172.21.15.85:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6406cf80 con 0x7f9c6406dad0 2022-12-01T09:40:18.574 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c637fe700 1 --2- 172.21.15.2:0/2346144345 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 0x7f9c64068e60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c6406d170 tx=0x7f9c4c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=799fb1dce02f45e2 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.574 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c627fc700 1 -- 172.21.15.2:0/2346144345 <== mon.1 v2:172.21.15.85:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c004020 con 0x7f9c6406dad0 2022-12-01T09:40:18.574 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c627fc700 1 -- 172.21.15.2:0/2346144345 <== mon.1 v2:172.21.15.85:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c004180 con 0x7f9c6406dad0 2022-12-01T09:40:18.574 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c627fc700 1 -- 172.21.15.2:0/2346144345 <== mon.1 v2:172.21.15.85:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c004530 con 0x7f9c6406dad0 2022-12-01T09:40:18.575 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c6a010700 1 -- 172.21.15.2:0/2346144345 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 msgr2=0x7f9c64068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.575 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c6a010700 1 --2- 172.21.15.2:0/2346144345 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 0x7f9c64068e60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9c6406d170 tx=0x7f9c4c009b20 comp rx=0 tx=0).stop 2022-12-01T09:40:18.575 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c6a010700 1 -- 172.21.15.2:0/2346144345 shutdown_connections 2022-12-01T09:40:18.576 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c6a010700 1 --2- 172.21.15.2:0/2346144345 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 0x7f9c64068e60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.576 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c6a010700 1 -- 172.21.15.2:0/2346144345 >> 172.21.15.2:0/2346144345 conn(0x7f9c640cbc00 msgr2=0x7f9c640cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:18.576 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c6a010700 1 -- 172.21.15.2:0/2346144345 shutdown_connections 2022-12-01T09:40:18.576 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.566+0000 7f9c6a010700 1 -- 172.21.15.2:0/2346144345 wait complete. 2022-12-01T09:40:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.568+0000 7f9c6a010700 1 Processor -- start 2022-12-01T09:40:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.568+0000 7f9c6a010700 1 -- start start 2022-12-01T09:40:18.580 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.568+0000 7f9c6a010700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c640693a0 0x7f9c6414c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.568+0000 7f9c6a010700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9c6406c940 0x7f9c640b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.568+0000 7f9c6a010700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 0x7f9c6415a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.568+0000 7f9c6a010700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6414fef0 con 0x7f9c640693a0 2022-12-01T09:40:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.568+0000 7f9c6a010700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b1fb0 con 0x7f9c6406dad0 2022-12-01T09:40:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.568+0000 7f9c6a010700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b2200 con 0x7f9c6406c940 2022-12-01T09:40:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c640693a0 0x7f9c6414c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c640693a0 0x7f9c6414c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33066/0 (socket says 172.21.15.2:33066) 2022-12-01T09:40:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 -- 172.21.15.2:0/447899710 learned_addr learned my addr 172.21.15.2:0/447899710 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c63fff700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 0x7f9c6415a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c62ffd700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9c6406c940 0x7f9c640b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 -- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9c6406c940 msgr2=0x7f9c640b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9c6406c940 0x7f9c640b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 -- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 msgr2=0x7f9c6415a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 0x7f9c6415a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 -- 172.21.15.2:0/447899710 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0094a0 con 0x7f9c640693a0 2022-12-01T09:40:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c62ffd700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9c6406c940 0x7f9c640b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:40:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c637fe700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c640693a0 0x7f9c6414c560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0090b0 tx=0x7f9c4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.585 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c5bfff700 1 -- 172.21.15.2:0/447899710 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c011e30 con 0x7f9c640693a0 2022-12-01T09:40:18.585 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c5bfff700 1 -- 172.21.15.2:0/447899710 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0036a0 con 0x7f9c640693a0 2022-12-01T09:40:18.585 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c640b24e0 con 0x7f9c640693a0 2022-12-01T09:40:18.585 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c5bfff700 1 -- 172.21.15.2:0/447899710 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c003a50 con 0x7f9c640693a0 2022-12-01T09:40:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.569+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c640b2aa0 con 0x7f9c640693a0 2022-12-01T09:40:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.570+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c64068b70 con 0x7f9c640693a0 2022-12-01T09:40:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.571+0000 7f9c5bfff700 1 -- 172.21.15.2:0/447899710 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f9c4c02a020 con 0x7f9c640693a0 2022-12-01T09:40:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.571+0000 7f9c5bfff700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9c48066590 0x7f9c48068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.571+0000 7f9c5bfff700 1 -- 172.21.15.2:0/447899710 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9c4c084590 con 0x7f9c640693a0 2022-12-01T09:40:18.587 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.571+0000 7f9c62ffd700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9c48066590 0x7f9c48068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.587 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.572+0000 7f9c62ffd700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9c48066590 0x7f9c48068a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c640cc1b0 tx=0x7f9c54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.587 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.576+0000 7f9c5bfff700 1 -- 172.21.15.2:0/447899710 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f9c4c051f70 con 0x7f9c640693a0 2022-12-01T09:40:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.811+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c6415c3c0 con 0x7f9c640693a0 2022-12-01T09:40:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.811+0000 7f9c5bfff700 1 -- 172.21.15.2:0/447899710 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3518 (secure 0 0 0) 0x7f9c4c055590 con 0x7f9c640693a0 2022-12-01T09:40:18.815 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:40:18.815 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6837/3331786962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6834","nonce":3331786962},{"type":"v1","addr":"172.21.15.85:6837","nonce":3331786962}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.85:6836/3185451792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6835","nonce":3185451792},{"type":"v1","addr":"172.21.15.85:6836","nonce":3185451792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6834/2016499744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":2016499744},{"type":"v1","addr":"172.21.15.2:6834","nonce":2016499744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1946406448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6833","nonce":1946406448},{"type":"v1","addr":"172.21.15.2:6835","nonce":1946406448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-12-01T09:40:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9c48066590 msgr2=0x7f9c48068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9c48066590 0x7f9c48068a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c640cc1b0 tx=0x7f9c54009040 comp rx=0 tx=0).stop 2022-12-01T09:40:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c640693a0 msgr2=0x7f9c6414c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c640693a0 0x7f9c6414c560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0090b0 tx=0x7f9c4c004750 comp rx=0 tx=0).stop 2022-12-01T09:40:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 shutdown_connections 2022-12-01T09:40:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f9c48066590 0x7f9c48068a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f9c6406dad0 0x7f9c6415a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c640693a0 0x7f9c6414c560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 --2- 172.21.15.2:0/447899710 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f9c6406c940 0x7f9c640b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 >> 172.21.15.2:0/447899710 conn(0x7f9c640cbc00 msgr2=0x7f9c6415b630 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 shutdown_connections 2022-12-01T09:40:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.813+0000 7f9c6a010700 1 -- 172.21.15.2:0/447899710 wait complete. 2022-12-01T09:40:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2022-12-01T09:40:18.829 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-12-01T09:40:18.953 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.950+0000 7f0f73bc4700 1 Processor -- start 2022-12-01T09:40:18.954 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.950+0000 7f0f73bc4700 1 -- start start 2022-12-01T09:40:18.956 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f73bc4700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0f6c0b2950 0x7f0f6c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.957 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f73bc4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.957 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f73bc4700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c0cc720 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.957 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f73bc4700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c159fc0 con 0x7f0f6c0b2950 2022-12-01T09:40:18.957 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f73bc4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c0cedc0 con 0x7f0f6c0cd2a0 2022-12-01T09:40:18.958 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7115f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0f6c0b2950 0x7f0f6c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.958 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7115f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0f6c0b2950 0x7f0f6c0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33076/0 (socket says 172.21.15.2:33076) 2022-12-01T09:40:18.958 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7115f700 1 -- 172.21.15.2:0/1690098410 learned_addr learned my addr 172.21.15.2:0/1690098410 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:18.958 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f72161700 1 --2- 172.21.15.2:0/1690098410 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.958 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7115f700 1 -- 172.21.15.2:0/1690098410 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 msgr2=0x7f0f6c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:18.959 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7115f700 1 --2- 172.21.15.2:0/1690098410 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c0cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.959 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7115f700 1 -- 172.21.15.2:0/1690098410 >> v1:172.21.15.85:6789/0 conn(0x7f0f6c0b5a60 legacy=0x7f0f6c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.959 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7115f700 1 -- 172.21.15.2:0/1690098410 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0cbdf0 con 0x7f0f6c0b2950 2022-12-01T09:40:18.959 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7115f700 1 --2- 172.21.15.2:0/1690098410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b2950 0x7f0f6c0ce7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0098f0 tx=0x7f0f5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a40c43e3b251226 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.960 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.951+0000 7f0f7095e700 1 -- 172.21.15.2:0/1690098410 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c011830 con 0x7f0f6c0b2950 2022-12-01T09:40:18.960 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f7095e700 1 -- 172.21.15.2:0/1690098410 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c011990 con 0x7f0f6c0b2950 2022-12-01T09:40:18.960 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f7095e700 1 -- 172.21.15.2:0/1690098410 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c011d20 con 0x7f0f6c0b2950 2022-12-01T09:40:18.960 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f73bc4700 1 -- 172.21.15.2:0/1690098410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b2950 msgr2=0x7f0f6c0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.960 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/1690098410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b2950 0x7f0f6c0ce7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c0098f0 tx=0x7f0f5c009c00 comp rx=0 tx=0).stop 2022-12-01T09:40:18.961 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f73bc4700 1 -- 172.21.15.2:0/1690098410 shutdown_connections 2022-12-01T09:40:18.961 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/1690098410 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.961 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/1690098410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b2950 0x7f0f6c0ce7c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.961 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f73bc4700 1 -- 172.21.15.2:0/1690098410 >> 172.21.15.2:0/1690098410 conn(0x7f0f6c064ec0 msgr2=0x7f0f6c065290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:18.962 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f73bc4700 1 -- 172.21.15.2:0/1690098410 shutdown_connections 2022-12-01T09:40:18.962 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.952+0000 7f0f73bc4700 1 -- 172.21.15.2:0/1690098410 wait complete. 2022-12-01T09:40:18.962 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.953+0000 7f0f73bc4700 1 Processor -- start 2022-12-01T09:40:18.962 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.953+0000 7f0f73bc4700 1 -- start start 2022-12-01T09:40:18.966 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f73bc4700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0f6c0b2950 0x7f0f6c06b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.967 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f73bc4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b5a60 0x7f0f6c0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.967 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f73bc4700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c1f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.967 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f73bc4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c0c3ce0 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.967 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f73bc4700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c06b8d0 con 0x7f0f6c0b2950 2022-12-01T09:40:18.968 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f73bc4700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1f2430 con 0x7f0f6c0cd2a0 2022-12-01T09:40:18.968 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f7115f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b5a60 0x7f0f6c0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.968 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f7115f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b5a60 0x7f0f6c0c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33082/0 (socket says 172.21.15.2:33082) 2022-12-01T09:40:18.968 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f7115f700 1 -- 172.21.15.2:0/109282992 learned_addr learned my addr 172.21.15.2:0/109282992 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:18.969 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f71960700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0f6c0b2950 0x7f0f6c06b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.969 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f72161700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c1f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.969 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f7115f700 1 -- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 msgr2=0x7f0f6c1f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.969 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f7115f700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c1f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.969 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f7115f700 1 -- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0f6c0b2950 msgr2=0x7f0f6c06b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:18.970 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f7115f700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0f6c0b2950 0x7f0f6c06b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:18.970 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f7115f700 1 -- 172.21.15.2:0/109282992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c009550 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.970 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.954+0000 7f0f72161700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c1f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:40:18.970 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.955+0000 7f0f7115f700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b5a60 0x7f0f6c0c46f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c000c00 tx=0x7f0f5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.971 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.955+0000 7f0f5a7fc700 1 -- 172.21.15.2:0/109282992 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c004480 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.971 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.955+0000 7f0f5a7fc700 1 -- 172.21.15.2:0/109282992 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c01f070 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.971 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.955+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c1f2710 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.971 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.955+0000 7f0f5a7fc700 1 -- 172.21.15.2:0/109282992 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c0038c0 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.971 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.955+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c0c41d0 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.972 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.956+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6c068b70 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.972 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.956+0000 7f0f5a7fc700 1 -- 172.21.15.2:0/109282992 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f0f5c02a020 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.972 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.956+0000 7f0f5a7fc700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f0f4c066590 0x7f0f4c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:18.972 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.957+0000 7f0f5a7fc700 1 -- 172.21.15.2:0/109282992 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0f5c084a60 con 0x7f0f6c0b5a60 2022-12-01T09:40:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.957+0000 7f0f71960700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f0f4c066590 0x7f0f4c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.957+0000 7f0f71960700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f0f4c066590 0x7f0f4c068a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f68007fd0 tx=0x7f0f68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:18.973 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:18.961+0000 7f0f5a7fc700 1 -- 172.21.15.2:0/109282992 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f0f5c0524c0 con 0x7f0f6c0b5a60 2022-12-01T09:40:19.155 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.152+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f6c158d00 con 0x7f0f6c0b5a60 2022-12-01T09:40:19.156 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.153+0000 7f0f5a7fc700 1 -- 172.21.15.2:0/109282992 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+13645 (secure 0 0 0) 0x7f0f5c015090 con 0x7f0f6c0b5a60 2022-12-01T09:40:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:40:19.157 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":20,"fsid":"3ffec3b9-dd5c-4bb9-b3a9-72bfb3e92803","created":"2022-12-01T09:39:56.873720+0000","modified":"2022-12-01T09:40:18.290647+0000","last_up_change":"2022-12-01T09:40:08.916618+0000","last_in_change":"2022-12-01T09:40:00.145489+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-12-01T09:40:08.277187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-12-01T09:40:16.135544+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"29ad9800-3b0e-4728-9c4b-27eed3978c17","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":101414},{"type":"v1","addr":"172.21.15.2:6801","nonce":101414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":101414},{"type":"v1","addr":"172.21.15.2:6803","nonce":101414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":101414},{"type":"v1","addr":"172.21.15.2:6807","nonce":101414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":101414},{"type":"v1","addr":"172.21.15.2:6805","nonce":101414}]},"public_addr":"172.21.15.2:6801/101414","cluster_addr":"172.21.15.2:6803/101414","heartbeat_back_addr":"172.21.15.2:6807/101414","heartbeat_front_addr":"172.21.15.2:6805/101414","state":["exists","up"]},{"osd":1,"uuid":"51463426-88ae-4c28-880b-1b655faff15c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":101416},{"type":"v1","addr":"172.21.15.2:6810","nonce":101416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":101416},{"type":"v1","addr":"172.21.15.2:6814","nonce":101416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":101416},{"type":"v1","addr":"172.21.15.2:6822","nonce":101416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":101416},{"type":"v1","addr":"172.21.15.2:6818","nonce":101416}]},"public_addr":"172.21.15.2:6810/101416","cluster_addr":"172.21.15.2:6814/101416","heartbeat_back_addr":"172.21.15.2:6822/101416","heartbeat_front_addr":"172.21.15.2:6818/101416","state":["exists","up"]},{"osd":2,"uuid":"817f70a8-c4b5-4470-beca-45061a2bf389","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6809","nonce":101415},{"type":"v1","addr":"172.21.15.2:6811","nonce":101415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6813","nonce":101415},{"type":"v1","addr":"172.21.15.2:6815","nonce":101415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6821","nonce":101415},{"type":"v1","addr":"172.21.15.2:6823","nonce":101415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6817","nonce":101415},{"type":"v1","addr":"172.21.15.2:6819","nonce":101415}]},"public_addr":"172.21.15.2:6811/101415","cluster_addr":"172.21.15.2:6815/101415","heartbeat_back_addr":"172.21.15.2:6823/101415","heartbeat_front_addr":"172.21.15.2:6819/101415","state":["exists","up"]},{"osd":3,"uuid":"379bf437-773e-4be5-ac5c-1a8c6b6663e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":101458},{"type":"v1","addr":"172.21.15.2:6825","nonce":101458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":101458},{"type":"v1","addr":"172.21.15.2:6827","nonce":101458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":101458},{"type":"v1","addr":"172.21.15.2:6831","nonce":101458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":101458},{"type":"v1","addr":"172.21.15.2:6829","nonce":101458}]},"public_addr":"172.21.15.2:6825/101458","cluster_addr":"172.21.15.2:6827/101458","heartbeat_back_addr":"172.21.15.2:6831/101458","heartbeat_front_addr":"172.21.15.2:6829/101458","state":["exists","up"]},{"osd":4,"uuid":"7ef04bad-dece-4880-9f9c-4ba20e210e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6808","nonce":101671},{"type":"v1","addr":"172.21.15.85:6809","nonce":101671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6810","nonce":101671},{"type":"v1","addr":"172.21.15.85:6811","nonce":101671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6814","nonce":101671},{"type":"v1","addr":"172.21.15.85:6815","nonce":101671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6812","nonce":101671},{"type":"v1","addr":"172.21.15.85:6813","nonce":101671}]},"public_addr":"172.21.15.85:6809/101671","cluster_addr":"172.21.15.85:6811/101671","heartbeat_back_addr":"172.21.15.85:6815/101671","heartbeat_front_addr":"172.21.15.85:6813/101671","state":["exists","up"]},{"osd":5,"uuid":"9f624f12-fff1-44e7-b4f2-76065630d976","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6800","nonce":101672},{"type":"v1","addr":"172.21.15.85:6801","nonce":101672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6802","nonce":101672},{"type":"v1","addr":"172.21.15.85:6803","nonce":101672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6806","nonce":101672},{"type":"v1","addr":"172.21.15.85:6807","nonce":101672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6804","nonce":101672},{"type":"v1","addr":"172.21.15.85:6805","nonce":101672}]},"public_addr":"172.21.15.85:6801/101672","cluster_addr":"172.21.15.85:6803/101672","heartbeat_back_addr":"172.21.15.85:6807/101672","heartbeat_front_addr":"172.21.15.85:6805/101672","state":["exists","up"]},{"osd":6,"uuid":"ecaf66b0-139a-4a68-8a19-914f382fee4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6824","nonce":101710},{"type":"v1","addr":"172.21.15.85:6825","nonce":101710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6826","nonce":101710},{"type":"v1","addr":"172.21.15.85:6827","nonce":101710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6830","nonce":101710},{"type":"v1","addr":"172.21.15.85:6831","nonce":101710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6828","nonce":101710},{"type":"v1","addr":"172.21.15.85:6829","nonce":101710}]},"public_addr":"172.21.15.85:6825/101710","cluster_addr":"172.21.15.85:6827/101710","heartbeat_back_addr":"172.21.15.85:6831/101710","heartbeat_front_addr":"172.21.15.85:6829/101710","state":["exists","up"]},{"osd":7,"uuid":"e3da96b3-89bd-463d-ad4c-f5fef6f5e5bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6816","nonce":101673},{"type":"v1","addr":"172.21.15.85:6817","nonce":101673}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6818","nonce":101673},{"type":"v1","addr":"172.21.15.85:6819","nonce":101673}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6822","nonce":101673},{"type":"v1","addr":"172.21.15.85:6823","nonce":101673}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.85:6820","nonce":101673},{"type":"v1","addr":"172.21.15.85:6821","nonce":101673}]},"public_addr":"172.21.15.85:6817/101673","cluster_addr":"172.21.15.85:6819/101673","heartbeat_back_addr":"172.21.15.85:6823/101673","heartbeat_front_addr":"172.21.15.85:6821/101673","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.388884+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.322448+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.322721+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.395855+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.392689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.409320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.377123+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:40:05.145226+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:40:19.158 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f0f4c066590 msgr2=0x7f0f4c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.159 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f0f4c066590 0x7f0f4c068a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f68007fd0 tx=0x7f0f68009040 comp rx=0 tx=0).stop 2022-12-01T09:40:19.159 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b5a60 msgr2=0x7f0f6c0c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.159 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b5a60 0x7f0f6c0c46f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c000c00 tx=0x7f0f5c004750 comp rx=0 tx=0).stop 2022-12-01T09:40:19.160 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 shutdown_connections 2022-12-01T09:40:19.160 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f0f4c066590 0x7f0f4c068a40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.160 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f0f6c0cd2a0 0x7f0f6c1f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.160 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f0f6c0b2950 0x7f0f6c06b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.160 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 --2- 172.21.15.2:0/109282992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f6c0b5a60 0x7f0f6c0c46f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.161 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 >> 172.21.15.2:0/109282992 conn(0x7f0f6c064ec0 msgr2=0x7f0f6c15c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:19.161 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 shutdown_connections 2022-12-01T09:40:19.161 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.155+0000 7f0f73bc4700 1 -- 172.21.15.2:0/109282992 wait complete. 2022-12-01T09:40:19.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-12-01T09:40:08.277187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-12-01T09:40:16.135544+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-12-01T09:40:19.171 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-12-01T09:40:19.298 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.295+0000 7f4a09bf3700 1 Processor -- start 2022-12-01T09:40:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.295+0000 7f4a09bf3700 1 -- start start 2022-12-01T09:40:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.296+0000 7f4a09bf3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a04162fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.296+0000 7f4a09bf3700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a040cde80 con 0x7f4a04158040 2022-12-01T09:40:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.296+0000 7f4a09bf3700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a040c8ed0 con 0x7f4a040c88f0 2022-12-01T09:40:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.296+0000 7f4a09bf3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040c91b0 con 0x7f4a0406ba70 2022-12-01T09:40:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.296+0000 7f4a037fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4a04158040 0x7f4a0415c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34210/0 (socket says 172.21.15.2:34210) 2022-12-01T09:40:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.296+0000 7f4a037fe700 1 -- 172.21.15.2:0/3803659931 learned_addr learned my addr 172.21.15.2:0/3803659931 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.296+0000 7f4a03fff700 1 --2- 172.21.15.2:0/3803659931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a04162fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.297+0000 7f4a03fff700 1 -- 172.21.15.2:0/3803659931 >> v1:172.21.15.85:6789/0 conn(0x7f4a040c88f0 legacy=0x7f4a0406b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.297+0000 7f4a03fff700 1 -- 172.21.15.2:0/3803659931 >> v1:172.21.15.2:6789/0 conn(0x7f4a04158040 legacy=0x7f4a0415c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.297+0000 7f4a03fff700 1 -- 172.21.15.2:0/3803659931 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040c9490 con 0x7f4a0406ba70 2022-12-01T09:40:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.297+0000 7f4a03fff700 1 --2- 172.21.15.2:0/3803659931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a04162fd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009870 tx=0x7f49fc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ace36ca65e019d server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:19.308 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.298+0000 7f4a027fc700 1 -- 172.21.15.2:0/3803659931 <== mon.2 v2:172.21.15.85:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc011830 con 0x7f4a0406ba70 2022-12-01T09:40:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.298+0000 7f4a027fc700 1 -- 172.21.15.2:0/3803659931 <== mon.2 v2:172.21.15.85:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc011990 con 0x7f4a0406ba70 2022-12-01T09:40:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.298+0000 7f4a027fc700 1 -- 172.21.15.2:0/3803659931 <== mon.2 v2:172.21.15.85:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc011d20 con 0x7f4a0406ba70 2022-12-01T09:40:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.299+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3803659931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 msgr2=0x7f4a04162fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.309 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.299+0000 7f4a09bf3700 1 --2- 172.21.15.2:0/3803659931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a04162fd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009870 tx=0x7f49fc009b80 comp rx=0 tx=0).stop 2022-12-01T09:40:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.299+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3803659931 shutdown_connections 2022-12-01T09:40:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.299+0000 7f4a09bf3700 1 --2- 172.21.15.2:0/3803659931 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a04162fd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.299+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3803659931 >> 172.21.15.2:0/3803659931 conn(0x7f4a040b0a40 msgr2=0x7f4a040ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:19.310 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.299+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3803659931 shutdown_connections 2022-12-01T09:40:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.299+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3803659931 wait complete. 2022-12-01T09:40:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.300+0000 7f4a09bf3700 1 Processor -- start 2022-12-01T09:40:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.300+0000 7f4a09bf3700 1 -- start start 2022-12-01T09:40:19.311 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a09bf3700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a040b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a09bf3700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4a040c88f0 0x7f4a04165530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a09bf3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a04158040 0x7f4a04165a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a09bf3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040b2d60 con 0x7f4a04158040 2022-12-01T09:40:19.312 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a09bf3700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04150840 con 0x7f4a040c88f0 2022-12-01T09:40:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a09bf3700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04150b00 con 0x7f4a0406ba70 2022-12-01T09:40:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a04158040 0x7f4a04165a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.313 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a04158040 0x7f4a04165a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33098/0 (socket says 172.21.15.2:33098) 2022-12-01T09:40:19.314 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 -- 172.21.15.2:0/3567338781 learned_addr learned my addr 172.21.15.2:0/3567338781 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:19.314 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a037fe700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a040b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.314 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a02ffd700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4a040c88f0 0x7f4a04165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.314 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 -- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 msgr2=0x7f4a040b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a040b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 -- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4a040c88f0 msgr2=0x7f4a04165530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4a040c88f0 0x7f4a04165530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 -- 172.21.15.2:0/3567338781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0094a0 con 0x7f4a04158040 2022-12-01T09:40:19.315 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f4a03fff700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a04158040 0x7f4a04165a70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49fc014040 tx=0x7f49fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.301+0000 7f49fbfff700 1 -- 172.21.15.2:0/3567338781 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0036a0 con 0x7f4a04158040 2022-12-01T09:40:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.302+0000 7f49fbfff700 1 -- 172.21.15.2:0/3567338781 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc01e070 con 0x7f4a04158040 2022-12-01T09:40:19.316 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.302+0000 7f49fbfff700 1 -- 172.21.15.2:0/3567338781 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc003a60 con 0x7f4a04158040 2022-12-01T09:40:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.302+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a04150de0 con 0x7f4a04158040 2022-12-01T09:40:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.302+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a041513a0 con 0x7f4a04158040 2022-12-01T09:40:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.303+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04068b70 con 0x7f4a04158040 2022-12-01T09:40:19.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.303+0000 7f49fbfff700 1 -- 172.21.15.2:0/3567338781 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f49fc00f050 con 0x7f4a04158040 2022-12-01T09:40:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.303+0000 7f49fbfff700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f49e8066590 0x7f49e8068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.303+0000 7f49fbfff700 1 -- 172.21.15.2:0/3567338781 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f49fc084430 con 0x7f4a04158040 2022-12-01T09:40:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.303+0000 7f4a037fe700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f49e8066590 0x7f49e8068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.318 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.304+0000 7f4a037fe700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f49e8066590 0x7f49e8068a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49f4005c90 tx=0x7f49f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:19.319 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.307+0000 7f49fbfff700 1 -- 172.21.15.2:0/3567338781 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f49fc051e90 con 0x7f4a04158040 2022-12-01T09:40:19.499 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.496+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a0406c0d0 con 0x7f4a04158040 2022-12-01T09:40:19.500 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.496+0000 7f49fbfff700 1 -- 172.21.15.2:0/3567338781 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49fc0554b0 con 0x7f4a04158040 2022-12-01T09:40:19.500 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2022-12-01T09:40:19.503 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f49e8066590 msgr2=0x7f49e8068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.503 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f49e8066590 0x7f49e8068a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49f4005c90 tx=0x7f49f4009040 comp rx=0 tx=0).stop 2022-12-01T09:40:19.504 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a04158040 msgr2=0x7f4a04165a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.504 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a04158040 0x7f4a04165a70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49fc014040 tx=0x7f49fc004750 comp rx=0 tx=0).stop 2022-12-01T09:40:19.504 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 shutdown_connections 2022-12-01T09:40:19.504 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f49e8066590 0x7f49e8068a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.505 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a04158040 0x7f4a04165a70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.505 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f4a0406ba70 0x7f4a040b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.505 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 --2- 172.21.15.2:0/3567338781 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f4a040c88f0 0x7f4a04165530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.506 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.498+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 >> 172.21.15.2:0/3567338781 conn(0x7f4a040b0a40 msgr2=0x7f4a040adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:19.506 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.499+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 shutdown_connections 2022-12-01T09:40:19.506 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.499+0000 7f4a09bf3700 1 -- 172.21.15.2:0/3567338781 wait complete. 2022-12-01T09:40:19.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-12-01T09:40:19.641 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.638+0000 7f5a696fd700 1 Processor -- start 2022-12-01T09:40:19.642 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.638+0000 7f5a696fd700 1 -- start start 2022-12-01T09:40:19.649 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a696fd700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f5a6415bb50 0x7f5a640cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.649 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a696fd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a640ce5d0 0x7f5a640b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.650 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a696fd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5a640b42d0 0x7f5a640697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.650 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a696fd700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640b4720 con 0x7f5a640b42d0 2022-12-01T09:40:19.650 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a696fd700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a640cd010 con 0x7f5a6415bb50 2022-12-01T09:40:19.650 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a696fd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640cca00 con 0x7f5a640ce5d0 2022-12-01T09:40:19.651 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a637fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5a640b42d0 0x7f5a640697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.651 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a637fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5a640b42d0 0x7f5a640697b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33106/0 (socket says 172.21.15.2:33106) 2022-12-01T09:40:19.652 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a637fe700 1 -- 172.21.15.2:0/1066786965 learned_addr learned my addr 172.21.15.2:0/1066786965 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:19.652 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a627fc700 1 --2- 172.21.15.2:0/1066786965 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a640ce5d0 0x7f5a640b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.652 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a62ffd700 1 --2- 172.21.15.2:0/1066786965 >> v2:172.21.15.85:3300/0 conn(0x7f5a6415bb50 0x7f5a640cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.652 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a637fe700 1 -- 172.21.15.2:0/1066786965 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a640ce5d0 msgr2=0x7f5a640b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:19.652 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a637fe700 1 --2- 172.21.15.2:0/1066786965 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a640ce5d0 0x7f5a640b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.653 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a637fe700 1 -- 172.21.15.2:0/1066786965 >> v2:172.21.15.85:3300/0 conn(0x7f5a6415bb50 msgr2=0x7f5a640cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:19.653 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a637fe700 1 --2- 172.21.15.2:0/1066786965 >> v2:172.21.15.85:3300/0 conn(0x7f5a6415bb50 0x7f5a640cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.653 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.639+0000 7f5a637fe700 1 -- 172.21.15.2:0/1066786965 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64160240 con 0x7f5a640b42d0 2022-12-01T09:40:19.653 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a637fe700 1 --2- 172.21.15.2:0/1066786965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640b42d0 0x7f5a640697b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a6406b680 tx=0x7f5a5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e0795bdeff9928 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:19.654 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a61ffb700 1 -- 172.21.15.2:0/1066786965 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00e070 con 0x7f5a640b42d0 2022-12-01T09:40:19.654 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a61ffb700 1 -- 172.21.15.2:0/1066786965 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00b910 con 0x7f5a640b42d0 2022-12-01T09:40:19.654 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a61ffb700 1 -- 172.21.15.2:0/1066786965 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00bcc0 con 0x7f5a640b42d0 2022-12-01T09:40:19.654 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 -- 172.21.15.2:0/1066786965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640b42d0 msgr2=0x7f5a640697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.654 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 --2- 172.21.15.2:0/1066786965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640b42d0 0x7f5a640697b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a6406b680 tx=0x7f5a5c00b330 comp rx=0 tx=0).stop 2022-12-01T09:40:19.655 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 -- 172.21.15.2:0/1066786965 shutdown_connections 2022-12-01T09:40:19.655 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 --2- 172.21.15.2:0/1066786965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640b42d0 0x7f5a640697b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.655 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 --2- 172.21.15.2:0/1066786965 >> v2:172.21.15.85:3300/0 conn(0x7f5a6415bb50 0x7f5a640cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.655 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 --2- 172.21.15.2:0/1066786965 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a640ce5d0 0x7f5a640b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.656 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 -- 172.21.15.2:0/1066786965 >> 172.21.15.2:0/1066786965 conn(0x7f5a640ced90 msgr2=0x7f5a640cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:19.656 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 -- 172.21.15.2:0/1066786965 shutdown_connections 2022-12-01T09:40:19.656 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.640+0000 7f5a696fd700 1 -- 172.21.15.2:0/1066786965 wait complete. 2022-12-01T09:40:19.656 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a696fd700 1 Processor -- start 2022-12-01T09:40:19.657 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a696fd700 1 -- start start 2022-12-01T09:40:19.657 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a696fd700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a640b42d0 0x7f5a641f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.657 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a696fd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640ce5d0 0x7f5a6414b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.657 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a696fd700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a6415bb50 0x7f5a64069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.658 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a696fd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415d9a0 con 0x7f5a640ce5d0 2022-12-01T09:40:19.658 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a696fd700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a641f5820 con 0x7f5a640b42d0 2022-12-01T09:40:19.658 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a696fd700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640c5f60 con 0x7f5a6415bb50 2022-12-01T09:40:19.658 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.642+0000 7f5a627fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640ce5d0 0x7f5a6414b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.658 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a627fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640ce5d0 0x7f5a6414b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33110/0 (socket says 172.21.15.2:33110) 2022-12-01T09:40:19.659 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a627fc700 1 -- 172.21.15.2:0/3688507658 learned_addr learned my addr 172.21.15.2:0/3688507658 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:19.659 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a637fe700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a6415bb50 0x7f5a64069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.659 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a62ffd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a640b42d0 0x7f5a641f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.660 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a627fc700 1 -- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a6415bb50 msgr2=0x7f5a64069cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.660 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a627fc700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a6415bb50 0x7f5a64069cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.660 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a627fc700 1 -- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a640b42d0 msgr2=0x7f5a641f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.660 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a627fc700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a640b42d0 0x7f5a641f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.661 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a627fc700 1 -- 172.21.15.2:0/3688507658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c009b30 con 0x7f5a640ce5d0 2022-12-01T09:40:19.661 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a62ffd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a640b42d0 0x7f5a641f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:40:19.661 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a627fc700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640ce5d0 0x7f5a6414b6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e930 tx=0x7f5a5400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:19.661 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a4b7fe700 1 -- 172.21.15.2:0/3688507658 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5400ee80 con 0x7f5a640ce5d0 2022-12-01T09:40:19.661 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a4b7fe700 1 -- 172.21.15.2:0/3688507658 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5400c850 con 0x7f5a640ce5d0 2022-12-01T09:40:19.662 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a641f17d0 con 0x7f5a640ce5d0 2022-12-01T09:40:19.662 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a4b7fe700 1 -- 172.21.15.2:0/3688507658 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5400cbe0 con 0x7f5a640ce5d0 2022-12-01T09:40:19.663 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.643+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a641f1d90 con 0x7f5a640ce5d0 2022-12-01T09:40:19.663 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.644+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6415db10 con 0x7f5a640ce5d0 2022-12-01T09:40:19.663 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.645+0000 7f5a4b7fe700 1 -- 172.21.15.2:0/3688507658 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f5a54019030 con 0x7f5a640ce5d0 2022-12-01T09:40:19.664 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.645+0000 7f5a4b7fe700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f5a44066590 0x7f5a44068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.664 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.645+0000 7f5a4b7fe700 1 -- 172.21.15.2:0/3688507658 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f5a54083310 con 0x7f5a640ce5d0 2022-12-01T09:40:19.664 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.645+0000 7f5a62ffd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f5a44066590 0x7f5a44068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.664 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.646+0000 7f5a62ffd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f5a44066590 0x7f5a44068a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5a640cc8c0 tx=0x7f5a58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:19.665 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.649+0000 7f5a4b7fe700 1 -- 172.21.15.2:0/3688507658 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f5a6415db10 con 0x7f5a640ce5d0 2022-12-01T09:40:19.841 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.837+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5a6415dae0 con 0x7f5a640ce5d0 2022-12-01T09:40:19.841 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.838+0000 7f5a4b7fe700 1 -- 172.21.15.2:0/3688507658 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a54050d70 con 0x7f5a640ce5d0 2022-12-01T09:40:19.841 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2022-12-01T09:40:19.843 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.839+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f5a44066590 msgr2=0x7f5a44068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.844 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f5a44066590 0x7f5a44068a40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5a640cc8c0 tx=0x7f5a58008040 comp rx=0 tx=0).stop 2022-12-01T09:40:19.844 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640ce5d0 msgr2=0x7f5a6414b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.844 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640ce5d0 0x7f5a6414b6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e930 tx=0x7f5a5400ec40 comp rx=0 tx=0).stop 2022-12-01T09:40:19.844 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 shutdown_connections 2022-12-01T09:40:19.845 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f5a44066590 0x7f5a44068a40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.845 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f5a6415bb50 0x7f5a64069cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.845 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f5a640b42d0 0x7f5a641f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.845 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 --2- 172.21.15.2:0/3688507658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a640ce5d0 0x7f5a6414b6d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.845 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 >> 172.21.15.2:0/3688507658 conn(0x7f5a640ced90 msgr2=0x7f5a640cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:19.846 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 shutdown_connections 2022-12-01T09:40:19.846 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.840+0000 7f5a696fd700 1 -- 172.21.15.2:0/3688507658 wait complete. 2022-12-01T09:40:19.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-12-01T09:40:19.980 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.977+0000 7fb82544e700 1 Processor -- start 2022-12-01T09:40:19.981 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.977+0000 7fb82544e700 1 -- start start 2022-12-01T09:40:19.984 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb82544e700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7fb82006dad0 0x7fb820068e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.984 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb82544e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb82006c940 0x7fb82015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.984 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb82544e700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82006cd40 con 0x7fb8200693a0 2022-12-01T09:40:19.985 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb82544e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb820069840 con 0x7fb82006c940 2022-12-01T09:40:19.985 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb82544e700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8200cde20 con 0x7fb82006dad0 2022-12-01T09:40:19.985 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81f7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb82006c940 0x7fb82015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.985 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81e7fc700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fb8200693a0 0x7fb82006c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34238/0 (socket says 172.21.15.2:34238) 2022-12-01T09:40:19.986 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81e7fc700 1 -- 172.21.15.2:0/148799186 learned_addr learned my addr 172.21.15.2:0/148799186 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:19.986 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81effd700 1 --2- 172.21.15.2:0/148799186 >> v2:172.21.15.85:3300/0 conn(0x7fb82006dad0 0x7fb820068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.986 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81f7fe700 1 -- 172.21.15.2:0/148799186 >> v2:172.21.15.85:3300/0 conn(0x7fb82006dad0 msgr2=0x7fb820068e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:19.986 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81f7fe700 1 --2- 172.21.15.2:0/148799186 >> v2:172.21.15.85:3300/0 conn(0x7fb82006dad0 0x7fb820068e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.987 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81f7fe700 1 -- 172.21.15.2:0/148799186 >> v1:172.21.15.2:6789/0 conn(0x7fb8200693a0 legacy=0x7fb82006c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.987 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81f7fe700 1 -- 172.21.15.2:0/148799186 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82006cf80 con 0x7fb82006c940 2022-12-01T09:40:19.987 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81dffb700 1 -- 172.21.15.2:0/148799186 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362526108 0 0) 0x7fb810001110 con 0x7fb8200693a0 2022-12-01T09:40:19.987 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81effd700 1 --2- 172.21.15.2:0/148799186 >> v2:172.21.15.85:3300/0 conn(0x7fb82006dad0 0x7fb820068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:40:19.988 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.978+0000 7fb81f7fe700 1 --2- 172.21.15.2:0/148799186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006c940 0x7fb82015f8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb82006d170 tx=0x7fb81800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b1bcd8cd03fbaf server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:19.988 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb81dffb700 1 -- 172.21.15.2:0/148799186 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818016070 con 0x7fb82006c940 2022-12-01T09:40:19.988 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb81dffb700 1 -- 172.21.15.2:0/148799186 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81800bba0 con 0x7fb82006c940 2022-12-01T09:40:19.988 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb81dffb700 1 -- 172.21.15.2:0/148799186 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81800e070 con 0x7fb82006c940 2022-12-01T09:40:19.988 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb82544e700 1 -- 172.21.15.2:0/148799186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006c940 msgr2=0x7fb82015f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb82544e700 1 --2- 172.21.15.2:0/148799186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006c940 0x7fb82015f8a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb82006d170 tx=0x7fb81800b330 comp rx=0 tx=0).stop 2022-12-01T09:40:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb82544e700 1 -- 172.21.15.2:0/148799186 shutdown_connections 2022-12-01T09:40:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb82544e700 1 --2- 172.21.15.2:0/148799186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006c940 0x7fb82015f8a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb82544e700 1 --2- 172.21.15.2:0/148799186 >> v2:172.21.15.85:3300/0 conn(0x7fb82006dad0 0x7fb820068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb82544e700 1 -- 172.21.15.2:0/148799186 >> 172.21.15.2:0/148799186 conn(0x7fb8200cbc00 msgr2=0x7fb8200cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb82544e700 1 -- 172.21.15.2:0/148799186 shutdown_connections 2022-12-01T09:40:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.979+0000 7fb82544e700 1 -- 172.21.15.2:0/148799186 wait complete. 2022-12-01T09:40:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb82544e700 1 Processor -- start 2022-12-01T09:40:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb82544e700 1 -- start start 2022-12-01T09:40:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb82544e700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb8200693a0 0x7fb8201f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb82544e700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb82006c940 0x7fb8200aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb82544e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006dad0 0x7fb8200b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb82544e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200b3820 con 0x7fb82006dad0 2022-12-01T09:40:19.995 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb82544e700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82015abc0 con 0x7fb8200693a0 2022-12-01T09:40:19.995 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb82544e700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82015aea0 con 0x7fb82006c940 2022-12-01T09:40:19.995 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb81f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006dad0 0x7fb8200b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.995 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb81f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006dad0 0x7fb8200b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33126/0 (socket says 172.21.15.2:33126) 2022-12-01T09:40:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.981+0000 7fb81f7fe700 1 -- 172.21.15.2:0/3319599229 learned_addr learned my addr 172.21.15.2:0/3319599229 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb81effd700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb8200693a0 0x7fb8201f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb81e7fc700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb82006c940 0x7fb8200aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb81f7fe700 1 -- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb82006c940 msgr2=0x7fb8200aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb81f7fe700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb82006c940 0x7fb8200aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb81f7fe700 1 -- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb8200693a0 msgr2=0x7fb8201f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb81f7fe700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb8200693a0 0x7fb8201f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb81f7fe700 1 -- 172.21.15.2:0/3319599229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb818009a70 con 0x7fb82006dad0 2022-12-01T09:40:19.998 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb81f7fe700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006dad0 0x7fb8200b3c40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b370 tx=0x7fb818004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:19.998 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb8077fe700 1 -- 172.21.15.2:0/3319599229 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818016040 con 0x7fb82006dad0 2022-12-01T09:40:19.998 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb8077fe700 1 -- 172.21.15.2:0/3319599229 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818003e10 con 0x7fb82006dad0 2022-12-01T09:40:19.998 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8200b3440 con 0x7fb82006dad0 2022-12-01T09:40:19.999 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb8077fe700 1 -- 172.21.15.2:0/3319599229 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81801b680 con 0x7fb82006dad0 2022-12-01T09:40:19.999 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.982+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8201f1270 con 0x7fb82006dad0 2022-12-01T09:40:19.999 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.983+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb820068b70 con 0x7fb82006dad0 2022-12-01T09:40:19.999 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.983+0000 7fb8077fe700 1 -- 172.21.15.2:0/3319599229 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fb818023030 con 0x7fb82006dad0 2022-12-01T09:40:20.000 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.984+0000 7fb8077fe700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fb800066590 0x7fb800068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:20.000 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.984+0000 7fb8077fe700 1 -- 172.21.15.2:0/3319599229 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb818084be0 con 0x7fb82006dad0 2022-12-01T09:40:20.000 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.984+0000 7fb81effd700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fb800066590 0x7fb800068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:20.000 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.984+0000 7fb81effd700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fb800066590 0x7fb800068a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb814006fd0 tx=0x7fb814008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:20.001 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:19.988+0000 7fb8077fe700 1 -- 172.21.15.2:0/3319599229 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7fb8180525e0 con 0x7fb82006dad0 2022-12-01T09:40:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.181+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb82015c3c0 con 0x7fb82006dad0 2022-12-01T09:40:20.185 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.181+0000 7fb8077fe700 1 -- 172.21.15.2:0/3319599229 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7fb818055c00 con 0x7fb82006dad0 2022-12-01T09:40:20.185 INFO:teuthology.orchestra.run.smithi002.stdout: 2022-12-01T09:40:20.186 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2022-12-01T09:40:20.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2022-12-01T09:40:20.186 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2022-12-01T09:40:20.186 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2022-12-01T09:40:20.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2022-12-01T09:40:20.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2022-12-01T09:40:20.187 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 2, 2022-12-01T09:40:20.188 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": "rbd" 2022-12-01T09:40:20.188 INFO:teuthology.orchestra.run.smithi002.stdout: } 2022-12-01T09:40:20.188 INFO:teuthology.orchestra.run.smithi002.stdout:] 2022-12-01T09:40:20.190 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fb800066590 msgr2=0x7fb800068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:20.190 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fb800066590 0x7fb800068a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb814006fd0 tx=0x7fb814008040 comp rx=0 tx=0).stop 2022-12-01T09:40:20.190 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006dad0 msgr2=0x7fb8200b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:20.190 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006dad0 0x7fb8200b3c40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b370 tx=0x7fb818004ad0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 shutdown_connections 2022-12-01T09:40:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7fb800066590 0x7fb800068a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.191 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb82006dad0 0x7fb8200b3c40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.192 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7fb8200693a0 0x7fb8201f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.192 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 --2- 172.21.15.2:0/3319599229 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7fb82006c940 0x7fb8200aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.192 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 >> 172.21.15.2:0/3319599229 conn(0x7fb8200cbc00 msgr2=0x7fb8200ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:20.193 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 shutdown_connections 2022-12-01T09:40:20.193 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.184+0000 7fb82544e700 1 -- 172.21.15.2:0/3319599229 wait complete. 2022-12-01T09:40:20.199 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-12-01T09:40:20.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-12-01T09:40:20.329 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.326+0000 7f37c1cfe700 1 Processor -- start 2022-12-01T09:40:20.329 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.326+0000 7f37c1cfe700 1 -- start start 2022-12-01T09:40:20.336 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.326+0000 7f37c1cfe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f37bc15bca0 0x7f37bc0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.326+0000 7f37c1cfe700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f37bc070900 0x7f37bc0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.326+0000 7f37c1cfe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc068e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.326+0000 7f37c1cfe700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc160860 con 0x7f37bc15bca0 2022-12-01T09:40:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.326+0000 7f37c1cfe700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc0ce3f0 con 0x7f37bc070900 2022-12-01T09:40:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.326+0000 7f37c1cfe700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc06fb40 con 0x7f37bc1602f0 2022-12-01T09:40:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f37bc15bca0 0x7f37bc0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f37bc15bca0 0x7f37bc0cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33130/0 (socket says 172.21.15.2:33130) 2022-12-01T09:40:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 -- 172.21.15.2:0/272965671 learned_addr learned my addr 172.21.15.2:0/272965671 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37baffd700 1 --2- 172.21.15.2:0/272965671 >> v2:172.21.15.85:3300/0 conn(0x7f37bc070900 0x7f37bc0cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bbfff700 1 --2- 172.21.15.2:0/272965671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 -- 172.21.15.2:0/272965671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 msgr2=0x7f37bc068e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 --2- 172.21.15.2:0/272965671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc068e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 -- 172.21.15.2:0/272965671 >> v2:172.21.15.85:3300/0 conn(0x7f37bc070900 msgr2=0x7f37bc0cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 --2- 172.21.15.2:0/272965671 >> v2:172.21.15.85:3300/0 conn(0x7f37bc070900 0x7f37bc0cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 -- 172.21.15.2:0/272965671 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc06fe20 con 0x7f37bc15bca0 2022-12-01T09:40:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37baffd700 1 --2- 172.21.15.2:0/272965671 >> v2:172.21.15.85:3300/0 conn(0x7f37bc070900 0x7f37bc0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:40:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bbfff700 1 --2- 172.21.15.2:0/272965671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:40:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37bb7fe700 1 --2- 172.21.15.2:0/272965671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc0cd970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0711a0 tx=0x7f37ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98656c0c739e08fa server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37ba7fc700 1 -- 172.21.15.2:0/272965671 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00b870 con 0x7f37bc15bca0 2022-12-01T09:40:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37ba7fc700 1 -- 172.21.15.2:0/272965671 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00b9d0 con 0x7f37bc15bca0 2022-12-01T09:40:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.327+0000 7f37ba7fc700 1 -- 172.21.15.2:0/272965671 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00bd60 con 0x7f37bc15bca0 2022-12-01T09:40:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 -- 172.21.15.2:0/272965671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 msgr2=0x7f37bc0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/272965671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc0cd970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0711a0 tx=0x7f37ac009bd0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 -- 172.21.15.2:0/272965671 shutdown_connections 2022-12-01T09:40:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/272965671 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc068e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/272965671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc0cd970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/272965671 >> v2:172.21.15.85:3300/0 conn(0x7f37bc070900 0x7f37bc0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 -- 172.21.15.2:0/272965671 >> 172.21.15.2:0/272965671 conn(0x7f37bc0ce880 msgr2=0x7f37bc0cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 -- 172.21.15.2:0/272965671 shutdown_connections 2022-12-01T09:40:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.328+0000 7f37c1cfe700 1 -- 172.21.15.2:0/272965671 wait complete. 2022-12-01T09:40:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.329+0000 7f37c1cfe700 1 Processor -- start 2022-12-01T09:40:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.329+0000 7f37c1cfe700 1 -- start start 2022-12-01T09:40:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37c1cfe700 1 --2- >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f37bc070900 0x7f37bc0ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37c1cfe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37c1cfe700 1 --2- >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc06d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37c1cfe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0adb10 con 0x7f37bc15bca0 2022-12-01T09:40:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37c1cfe700 1 -- --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0ad270 con 0x7f37bc070900 2022-12-01T09:40:20.346 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37c1cfe700 1 -- --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0ad550 con 0x7f37bc1602f0 2022-12-01T09:40:20.346 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:20.346 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc06cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33136/0 (socket says 172.21.15.2:33136) 2022-12-01T09:40:20.346 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 -- 172.21.15.2:0/2064310007 learned_addr learned my addr 172.21.15.2:0/2064310007 (peer_addr_for_me v2:172.21.15.2:0/0) 2022-12-01T09:40:20.347 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37bb7fe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f37bc070900 0x7f37bc0ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:20.347 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37bbfff700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:20.347 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 -- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 msgr2=0x7f37bc06d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:20.347 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc06d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.348 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 -- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f37bc070900 msgr2=0x7f37bc0ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:20.348 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f37bc070900 0x7f37bc0ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:20.348 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 -- 172.21.15.2:0/2064310007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac009550 con 0x7f37bc15bca0 2022-12-01T09:40:20.348 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37bb7fe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f37bc070900 0x7f37bc0ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:40:20.349 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.330+0000 7f37baffd700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc06cbd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f37b000d7d0 tx=0x7f37b000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:20.349 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.331+0000 7f37a3fff700 1 -- 172.21.15.2:0/2064310007 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0004030 con 0x7f37bc15bca0 2022-12-01T09:40:20.349 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.331+0000 7f37a3fff700 1 -- 172.21.15.2:0/2064310007 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b0004190 con 0x7f37bc15bca0 2022-12-01T09:40:20.349 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.331+0000 7f37a3fff700 1 -- 172.21.15.2:0/2064310007 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0004540 con 0x7f37bc15bca0 2022-12-01T09:40:20.349 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.331+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc0ad830 con 0x7f37bc15bca0 2022-12-01T09:40:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.331+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc142af0 con 0x7f37bc15bca0 2022-12-01T09:40:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.332+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc04e190 con 0x7f37bc15bca0 2022-12-01T09:40:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.332+0000 7f37a3fff700 1 -- 172.21.15.2:0/2064310007 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f37b001a020 con 0x7f37bc15bca0 2022-12-01T09:40:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.332+0000 7f37a3fff700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f379c066590 0x7f379c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.333+0000 7f37a3fff700 1 -- 172.21.15.2:0/2064310007 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f37b00839a0 con 0x7f37bc15bca0 2022-12-01T09:40:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.333+0000 7f37bb7fe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f379c066590 0x7f379c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.333+0000 7f37bb7fe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f379c066590 0x7f379c068a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37bc13f540 tx=0x7f37ac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:40:20.352 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.337+0000 7f37a3fff700 1 -- 172.21.15.2:0/2064310007 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f37b0051310 con 0x7f37bc15bca0 2022-12-01T09:40:20.531 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:20.528+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f37bc070100 con 0x7f37bc15bca0 2022-12-01T09:40:21.309 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.306+0000 7f37a3fff700 1 -- 172.21.15.2:0/2064310007 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7f37b0054930 con 0x7f37bc15bca0 2022-12-01T09:40:21.315 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f379c066590 msgr2=0x7f379c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:21.316 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f379c066590 0x7f379c068a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37bc13f540 tx=0x7f37ac008030 comp rx=0 tx=0).stop 2022-12-01T09:40:21.316 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 msgr2=0x7f37bc06cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:21.316 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc06cbd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f37b000d7d0 tx=0x7f37b000db90 comp rx=0 tx=0).stop 2022-12-01T09:40:21.316 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 shutdown_connections 2022-12-01T09:40:21.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:6832/101005,v1:172.21.15.85:6833/101005] conn(0x7f379c066590 0x7f379c068a40 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37bc13f540 tx=0x7f37ac008030 comp rx=0 tx=0).stop 2022-12-01T09:40:21.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f37bc1602f0 0x7f37bc06d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:21.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f37bc070900 0x7f37bc0ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:21.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 --2- 172.21.15.2:0/2064310007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37bc15bca0 0x7f37bc06cbd0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f37b000d7d0 tx=0x7f37b000db90 comp rx=0 tx=0).stop 2022-12-01T09:40:21.317 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.311+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 >> 172.21.15.2:0/2064310007 conn(0x7f37bc0ce880 msgr2=0x7f37bc15aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:40:21.318 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.312+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 shutdown_connections 2022-12-01T09:40:21.318 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.312+0000 7f37c1cfe700 1 -- 172.21.15.2:0/2064310007 wait complete. 2022-12-01T09:40:21.318 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2022-12-01T09:40:21.332 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2022-12-01T09:40:21.469 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.466+0000 7f30afcf2700 1 Processor -- start 2022-12-01T09:40:21.470 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.466+0000 7f30afcf2700 1 -- start start 2022-12-01T09:40:21.476 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30afcf2700 1 --2- >> v2:172.21.15.85:3300/0 conn(0x7f30a80b1740 0x7f30a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30afcf2700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a80b1be0 con 0x7f30a806fae0 2022-12-01T09:40:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30afcf2700 1 -- --> v1:172.21.15.85:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a806d980 con 0x7f30a8159dd0 2022-12-01T09:40:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30afcf2700 1 -- --> v2:172.21.15.85:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a806d0e0 con 0x7f30a80b1740 2022-12-01T09:40:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30ada8e700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f30a806fae0 0x7f30a806feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:34264/0 (socket says 172.21.15.2:34264) 2022-12-01T09:40:21.477 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30ada8e700 1 -- 172.21.15.2:0/3802929037 learned_addr learned my addr 172.21.15.2:0/3802929037 (peer_addr_for_me v1:172.21.15.2:0/0) 2022-12-01T09:40:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32301153 0 0) 0x7f30a80b1be0 con 0x7f30a806fae0 2022-12-01T09:40:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30ae28f700 1 --2- 172.21.15.2:0/3802929037 >> v2:172.21.15.85:3300/0 conn(0x7f30a80b1740 0x7f30a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.467+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30900035b0 con 0x7f30a806fae0 2022-12-01T09:40:21.478 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 360751634 0 0) 0x7f30900035b0 con 0x7f30a806fae0 2022-12-01T09:40:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 >> v2:172.21.15.85:3300/0 conn(0x7f30a80b1740 msgr2=0x7f30a8069420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 --2- 172.21.15.2:0/3802929037 >> v2:172.21.15.85:3300/0 conn(0x7f30a80b1740 0x7f30a8069420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 >> v1:172.21.15.85:6789/0 conn(0x7f30a8159dd0 legacy=0x7f30a80b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a806d3c0 con 0x7f30a806fae0 2022-12-01T09:40:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2890946737 0 0) 0x7f3098001670 con 0x7f30a806fae0 2022-12-01T09:40:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 >> v1:172.21.15.2:6789/0 conn(0x7f30a806fae0 legacy=0x7f30a806feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:40:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 --2- 172.21.15.2:0/3802929037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3090003a00 0x7f3090003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 --2- 172.21.15.2:0/3802929037 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3090004850 0x7f3090004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 --2- 172.21.15.2:0/3802929037 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3090005200 0x7f309000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:40:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a806d0e0 con 0x7f3090003a00 2022-12-01T09:40:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 --> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309000c080 con 0x7f3090005200 2022-12-01T09:40:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30aca8c700 1 -- 172.21.15.2:0/3802929037 --> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309000c240 con 0x7f3090004850 2022-12-01T09:40:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30ada8e700 1 --2- 172.21.15.2:0/3802929037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3090003a00 0x7f3090003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30ae28f700 1 --2- 172.21.15.2:0/3802929037 >> [v2:172.21.15.85:3300/0,v1:172.21.15.85:6789/0] conn(0x7f3090005200 0x7f309000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30ad28d700 1 --2- 172.21.15.2:0/3802929037 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3090004850 0x7f3090004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:40:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30ada8e700 1 -- 172.21.15.2:0/3802929037 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3090004850 msgr2=0x7f3090004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:40:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7f30ada8e700 1 --2- 172.21.15.2:0/3802929037 >> [v2:172.21.15.85:3301/0,v1:172.21.15.85:6790/0] conn(0x7f3090004850 0x7f3090004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:40:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2022-12-01T09:40:21.468+0000 7