2022-12-01T09:36:13.947 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-12-01T09:36:13.949 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090979 --owner scheduled_khiremat@teuthology --verbose --name khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} -- /home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090979/orig.config.yaml 2022-12-01T09:36:14.007 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-12-01T09:36:14.094 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090979 branch: wip-khiremat-47893-test-mgr-finisher-thread-5 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7090979' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-khiremat-47893-test-mgr-finisher-thread-5 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-khiremat-47893-test-mgr-finisher-thread-5 sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 owner: scheduled_khiremat@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 sleep_before_teardown: 0 suite: fs:volumes suite_branch: wip-khiremat-47893-test-mgr-finisher-thread-5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f6922ca21f63dad2b198f92826f75d268f12fc97/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaq8FNWQ+dGpbsVxrhRBQus5ucOH55AdmRfW6SAsawZe5S7f5ymoeSxB4tLgMGB5elAIJb3WSTwVg5JQU+vSXsqzVUNzvMpu0Ifz2xMA2xsN3k6Ql3bQhtsKGDU+96vHMsk2JZXQBlROzbtbpy9Jpm6vMzBGyWZc91VUzaHZkGlaMPdEQ7NrgcGkX9DdI1x3XmUM2D1TUUrQXPQrY4y5W2MOt7dGQi7hEjkuabvVxHcqq+dnZ/Nyol3JUJ6lupNzrjPv1QvkY3XUiNcXdyF9KzUWVcx+gyGrgivzj21+U+snkaEnR08iAAfIq/eL4UBmy/AFE65Pf/yWQSy32JkzEhnES+p29Y6L8nKTQNXAWUyxS9kuAYTYoabajXtABhtyzeyr0qU51u1OyMHafTWst9frJjgV7nRfhS6ZLy2NLA8b0LLzHKSYgFpWXp0agnRNOepVkKF3hME5D8VETygQGmJ0y6mBZUQB2ImlRYtKccCOc4+fGl4FR6TtNk+6xkqgE= smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOn05x+aW6hZoPcIlnRTFEqP2h+sLA6MWRf8oZN7upq8Sh1LJtLs8NIBUk77CYZKou7JMWol2TDqqj48SjnrZACgAGmi1oW1suwpw5iYrtRWTIDarYptaFNI/OLDsMjPkEwkkdibeh0rtVRin3J2CAdYud5+m0S9w5dGFfHWouTNsGNuKMkakw/FwMcdDujGCh5QqMeVmbSxV+IZr1w6zeCgthzU4+mDjFpiyaNfoltuvM+2M0OuU0QwM2Osl0hvMCbHRb555xtz66jArweFFcn4PiCX5ati4LidH5bwSJyxMvQjcqPH4xFYa2kX7+84zYLIaW1AH0fiS9oc9dolc0OVsoXlW4qRvAGYkwA+4PB8D76+jPwj//lQpuiU9Q8sZIlHZ79GHwG2+3qLPFGEsglv4eZDP3jt61BKF752g8No+2bBKbWE1HFmH4d/J1E8lMC4HEajx1p3AL5xVmCwAX4Pb++YQ3TOPmyI+c+a++IdygVGlgQPQxpCym+zqTyvc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-24_05:02:42 tube: smithi user: khiremat verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23126 2022-12-01T09:36:14.094 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f6922ca21f63dad2b198f92826f75d268f12fc97/qa; will attempt to use it 2022-12-01T09:36:14.095 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f6922ca21f63dad2b198f92826f75d268f12fc97/qa/tasks 2022-12-01T09:36:14.095 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-12-01T09:36:14.098 INFO:teuthology.task.internal:Checking packages... 2022-12-01T09:36:14.121 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'f6922ca21f63dad2b198f92826f75d268f12fc97' 2022-12-01T09:36:14.122 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-12-01T09:36:14.122 INFO:teuthology.packaging:ref: None 2022-12-01T09:36:14.123 INFO:teuthology.packaging:tag: None 2022-12-01T09:36:14.123 INFO:teuthology.packaging:branch: wip-khiremat-47893-test-mgr-finisher-thread-5 2022-12-01T09:36:14.123 INFO:teuthology.packaging:sha1: f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:36:14.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-khiremat-47893-test-mgr-finisher-thread-5 2022-12-01T09:36:14.450 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-676.gf6922ca2 2022-12-01T09:36:14.450 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-12-01T09:36:14.458 INFO:teuthology.task.internal:no buildpackages task found 2022-12-01T09:36:14.458 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-12-01T09:36:14.468 INFO:teuthology.task.internal:Saving configuration 2022-12-01T09:36:14.508 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-12-01T09:36:14.514 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-12-01T09:36:14.539 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090979', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-12-01 09:31:07.788677', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFMsIeVT42g8UutdeDyinorcVvI6vdLbT06th6IfqjgVWesGpk0ke06hTrSZUVFkbJjD3sRnO6C4M4JQexmFUzA='} 2022-12-01T09:36:14.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090979', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-12-01 09:31:07.790195', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEn4AWExaCVvbM9+5ZuqMMJ/ro7jYzsjW7admivLISFNhBZ0oxHeZZwXqPspRiA14eBq+zNnMioBcRWuXsWnvtY='} 2022-12-01T09:36:14.564 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-12-01T09:36:14.571 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-12-01T09:36:14.571 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-12-01T09:36:14.571 INFO:teuthology.run_tasks:Running task console_log... 2022-12-01T09:36:14.648 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fee0bf8e8c8>, signals=[15]) 2022-12-01T09:36:14.650 INFO:teuthology.run_tasks:Running task internal.connect... 2022-12-01T09:36:14.662 INFO:teuthology.task.internal:Opening connections... 2022-12-01T09:36:14.663 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2022-12-01T09:36:14.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-12-01T09:36:14.751 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2022-12-01T09:36:14.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-12-01T09:36:14.830 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-12-01T09:36:14.837 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2022-12-01T09:36:14.859 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2022-12-01T09:36:14.859 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2022-12-01T09:36:14.918 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Red Hat Enterprise Linux" 2022-12-01T09:36:14.919 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8.6 (Ootpa)" 2022-12-01T09:36:14.919 INFO:teuthology.orchestra.run.smithi084.stdout:ID="rhel" 2022-12-01T09:36:14.919 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="fedora" 2022-12-01T09:36:14.919 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8.6" 2022-12-01T09:36:14.920 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8" 2022-12-01T09:36:14.920 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-12-01T09:36:14.920 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2022-12-01T09:36:14.920 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-12-01T09:36:14.921 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.redhat.com/" 2022-12-01T09:36:14.921 INFO:teuthology.orchestra.run.smithi084.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-12-01T09:36:14.921 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-12-01T09:36:14.921 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:36:14.921 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-12-01T09:36:14.922 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-12-01T09:36:14.922 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-12-01T09:36:14.922 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-12-01T09:36:14.923 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2022-12-01T09:36:14.944 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2022-12-01T09:36:14.965 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2022-12-01T09:36:14.965 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2022-12-01T09:36:15.024 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux" 2022-12-01T09:36:15.024 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.6 (Ootpa)" 2022-12-01T09:36:15.025 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel" 2022-12-01T09:36:15.025 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora" 2022-12-01T09:36:15.025 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.6" 2022-12-01T09:36:15.025 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2022-12-01T09:36:15.026 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-12-01T09:36:15.026 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2022-12-01T09:36:15.026 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-12-01T09:36:15.026 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/" 2022-12-01T09:36:15.027 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-12-01T09:36:15.027 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-12-01T09:36:15.027 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:36:15.027 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-12-01T09:36:15.028 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-12-01T09:36:15.028 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-12-01T09:36:15.028 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-12-01T09:36:15.029 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2022-12-01T09:36:15.050 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-12-01T09:36:15.063 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-12-01T09:36:15.072 INFO:teuthology.task.internal:Checking for old test directory... 2022-12-01T09:36:15.073 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2022-12-01T09:36:15.077 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2022-12-01T09:36:15.096 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-12-01T09:36:15.102 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-12-01T09:36:15.103 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2022-12-01T09:36:15.135 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2022-12-01T09:36:15.177 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-12-01T09:36:15.235 INFO:teuthology.run_tasks:Running task kernel... 2022-12-01T09:36:15.249 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-12-01T09:36:15.250 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-12-01T09:36:15.251 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2022-12-01T09:36:15.252 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2022-12-01T09:36:15.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-12-01T09:36:15.271 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2022-12-01T09:36:15.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-12-01T09:36:15.273 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2022-12-01T09:36:15.329 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-372.9.1.el8.x86_64 2022-12-01T09:36:15.330 INFO:teuthology.task.kernel:Running kernel on smithi191: 4.18.0-372.9.1.el8.x86_64 2022-12-01T09:36:15.330 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2022-12-01T09:36:15.332 INFO:teuthology.orchestra.run.smithi084.stdout:4.18.0-372.9.1.el8.x86_64 2022-12-01T09:36:15.332 INFO:teuthology.task.kernel:Running kernel on smithi084: 4.18.0-372.9.1.el8.x86_64 2022-12-01T09:36:15.333 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum install -y kernel 2022-12-01T09:36:16.337 INFO:teuthology.orchestra.run.smithi084.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:16.337 INFO:teuthology.orchestra.run.smithi084.stdout:Unable to read consumer identity 2022-12-01T09:36:16.344 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:16.344 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2022-12-01T09:36:16.414 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:36:16.414 INFO:teuthology.orchestra.run.smithi084.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:36:16.415 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:36:16.434 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:36:16.435 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:36:16.435 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:36:19.235 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 2.3 kB/s | 3.5 kB 00:01 2022-12-01T09:36:19.247 INFO:teuthology.orchestra.run.smithi084.stdout:Copr repo for python3-asyncssh owned by ceph 2.1 kB/s | 3.5 kB 00:01 2022-12-01T09:36:22.463 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 13 MB 00:03 2022-12-01T09:36:25.496 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 415 kB/s | 24 kB 00:00 2022-12-01T09:36:26.199 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-12-01T09:36:26.266 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2022-12-01T09:36:26.267 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2022-12-01T09:36:26.267 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2022-12-01T09:36:26.368 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 1.8 MB/s | 13 MB 00:07 2022-12-01T09:36:26.432 DEBUG:teuthology.orchestra.run.smithi084:> echo no | sudo yum reinstall kernel || true 2022-12-01T09:36:26.764 INFO:teuthology.orchestra.run.smithi084.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:26.765 INFO:teuthology.orchestra.run.smithi084.stdout:Unable to read consumer identity 2022-12-01T09:36:26.823 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:36:26.824 INFO:teuthology.orchestra.run.smithi084.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:36:26.824 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:36:27.062 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:02 ago on Thu 01 Dec 2022 09:36:25 AM UTC. 2022-12-01T09:36:27.113 INFO:teuthology.orchestra.run.smithi084.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-12-01T09:36:27.145 INFO:teuthology.orchestra.run.smithi084.stderr:Error: No packages marked for reinstall. 2022-12-01T09:36:27.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum reinstall -y kernel || true 2022-12-01T09:36:27.523 INFO:teuthology.orchestra.run.smithi084.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:27.523 INFO:teuthology.orchestra.run.smithi084.stdout:Unable to read consumer identity 2022-12-01T09:36:27.576 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:36:27.577 INFO:teuthology.orchestra.run.smithi084.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:36:27.577 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:36:27.813 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:02 ago on Thu 01 Dec 2022 09:36:25 AM UTC. 2022-12-01T09:36:27.851 INFO:teuthology.orchestra.run.smithi084.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-12-01T09:36:27.899 INFO:teuthology.orchestra.run.smithi084.stderr:Error: No packages marked for reinstall. 2022-12-01T09:36:27.949 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q kernel --last | head -n 1 2022-12-01T09:36:28.029 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-12-01T09:36:28.030 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-12-01T09:36:28.031 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-12-01T09:36:28.031 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-12-01T09:36:28.032 DEBUG:teuthology.orchestra.run.smithi084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-12-01T09:36:28.063 INFO:teuthology.orchestra.run.smithi084.stdout:ttyS1 2022-12-01T09:36:28.075 DEBUG:teuthology.parallel:result is None 2022-12-01T09:36:29.360 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 425 kB/s | 24 kB 00:00 2022-12-01T09:36:30.030 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-12-01T09:36:30.099 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2022-12-01T09:36:30.100 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2022-12-01T09:36:30.101 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2022-12-01T09:36:30.245 DEBUG:teuthology.orchestra.run.smithi191:> echo no | sudo yum reinstall kernel || true 2022-12-01T09:36:30.575 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:30.576 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2022-12-01T09:36:30.635 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:36:30.635 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:36:30.636 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:36:30.879 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:01 ago on Thu 01 Dec 2022 09:36:29 AM UTC. 2022-12-01T09:36:30.922 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-12-01T09:36:30.956 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2022-12-01T09:36:31.004 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum reinstall -y kernel || true 2022-12-01T09:36:31.333 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-12-01T09:36:31.334 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2022-12-01T09:36:31.383 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:36:31.383 INFO:teuthology.orchestra.run.smithi191.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-12-01T09:36:31.384 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:36:31.626 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:02 ago on Thu 01 Dec 2022 09:36:29 AM UTC. 2022-12-01T09:36:31.662 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-12-01T09:36:31.683 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2022-12-01T09:36:31.730 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel --last | head -n 1 2022-12-01T09:36:31.810 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-12-01T09:36:31.812 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-12-01T09:36:31.812 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-12-01T09:36:31.813 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-12-01T09:36:31.813 DEBUG:teuthology.orchestra.run.smithi191:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-12-01T09:36:31.844 INFO:teuthology.orchestra.run.smithi191.stdout:ttyS1 2022-12-01T09:36:31.857 DEBUG:teuthology.parallel:result is None 2022-12-01T09:36:31.858 INFO:teuthology.run_tasks:Running task internal.base... 2022-12-01T09:36:31.867 INFO:teuthology.task.internal:Creating test directory... 2022-12-01T09:36:31.868 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-12-01T09:36:31.871 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-12-01T09:36:31.915 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-12-01T09:36:31.922 INFO:teuthology.run_tasks:Running task internal.archive... 2022-12-01T09:36:31.929 INFO:teuthology.task.internal:Creating archive directory... 2022-12-01T09:36:31.930 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-12-01T09:36:31.932 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-12-01T09:36:31.987 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-12-01T09:36:31.995 INFO:teuthology.task.internal:Enabling coredump saving... 2022-12-01T09:36:31.996 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-12-01T09:36:31.999 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-12-01T09:36:32.027 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-12-01T09:36:32.039 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-12-01T09:36:32.054 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-12-01T09:36:32.064 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-12-01T09:36:32.066 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-12-01T09:36:32.073 INFO:teuthology.task.internal:Configuring sudo... 2022-12-01T09:36:32.073 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-12-01T09:36:32.083 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-12-01T09:36:32.133 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-12-01T09:36:32.142 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-12-01T09:36:32.143 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-12-01T09:36:32.153 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-12-01T09:36:32.189 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-12-01T09:36:32.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-12-01T09:36:32.313 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-12-01T09:36:32.358 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-12-01T09:36:32.399 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:36:32.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-12-01T09:36:32.467 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-12-01T09:36:32.511 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-12-01T09:36:32.560 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-12-01T09:36:32.642 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-12-01T09:36:32.685 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:36:32.686 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-12-01T09:36:32.751 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2022-12-01T09:36:32.754 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2022-12-01T09:36:32.802 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-12-01T09:36:32.840 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-12-01T09:36:33.316 INFO:teuthology.run_tasks:Running task internal.timer... 2022-12-01T09:36:33.325 INFO:teuthology.task.internal:Starting timer... 2022-12-01T09:36:33.326 INFO:teuthology.run_tasks:Running task pcp... 2022-12-01T09:36:33.339 INFO:teuthology.run_tasks:Running task selinux... 2022-12-01T09:36:33.349 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2022-12-01T09:36:33.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2022-12-01T09:36:33.409 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2022-12-01T09:36:33.411 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2022-12-01T09:36:33.468 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2022-12-01T09:36:33.470 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-12-01T09:36:33.470 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2022-12-01T09:36:33.496 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2022-12-01T09:36:33.497 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2022-12-01T09:36:33.523 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2022-12-01T09:36:33.523 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi084.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2022-12-01T09:36:33.524 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-12-01T09:36:33.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-12-01T09:36:33.566 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-12-01T09:36:33.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-12-01T09:36:33.593 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-12-01T09:36:33.594 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2022-12-01T09:36:33.633 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2022-12-01T09:36:33.662 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-12-01T09:36:33.672 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2022-12-01T09:36:33.680 INFO:teuthology.repo_utils:Fetching main from origin 2022-12-01T09:36:33.824 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-12-01T09:36:33.844 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-12-01T09:36:33.846 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi084.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-12-01T09:44:23.785 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi084.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2022-12-01T09:44:23.788 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2022-12-01T09:44:23.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-12-01T09:44:23.877 DEBUG:teuthology.orchestra.run.smithi084:> true 2022-12-01T09:44:23.945 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2022-12-01T09:44:23.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-12-01T09:44:24.023 DEBUG:teuthology.orchestra.run.smithi191:> true 2022-12-01T09:44:24.090 INFO:teuthology.run_tasks:Running task clock... 2022-12-01T09:44:24.106 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-12-01T09:44:24.107 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-12-01T09:44:24.111 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-12-01T09:44:24.114 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-12-01T09:44:24.115 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-12-01T09:44:24.145 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-12-01T09:44:24.160 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-12-01T09:44:24.161 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-12-01T09:44:24.174 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-12-01T09:44:24.225 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2022-12-01T09:44:24.236 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2022-12-01T09:44:24.244 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2022-12-01T09:44:24.250 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-12-01T09:44:24.255 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2022-12-01T09:44:24.264 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-12-01T09:44:24.270 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-12-01T09:44:24.285 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-12-01T09:44:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:bash: ntpq: command not found 2022-12-01T09:44:24.332 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-12-01T09:44:24.333 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2022-12-01T09:44:24.333 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:44:24.333 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:44:24.333 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:44:24.334 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:44:24.347 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2022-12-01T09:44:24.351 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-12-01T09:44:24.351 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2022-12-01T09:44:24.351 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:44:24.352 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:44:24.352 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:44:24.352 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-12-01T09:44:24.353 INFO:teuthology.run_tasks:Running task install... 2022-12-01T09:44:24.366 DEBUG:teuthology.task.install:project ceph 2022-12-01T09:44:24.366 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}} 2022-12-01T09:44:24.367 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'} 2022-12-01T09:44:24.367 INFO:teuthology.task.install:Using flavor: default 2022-12-01T09:44:24.374 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-12-01T09:44:24.375 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-12-01T09:44:24.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo lsb_release -is 2022-12-01T09:44:24.417 INFO:teuthology.orchestra.run.smithi084.stdout:RedHatEnterprise 2022-12-01T09:44:24.418 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:44:24.419 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-12-01T09:44:24.420 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97', 'tag': None, 'wait_for_package': False} 2022-12-01T09:44:24.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo lsb_release -is 2022-12-01T09:44:24.462 INFO:teuthology.orchestra.run.smithi191.stdout:RedHatEnterprise 2022-12-01T09:44:24.463 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:44:24.464 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97', 'tag': None, 'wait_for_package': False} 2022-12-01T09:44:24.464 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-12-01T09:44:24.499 INFO:teuthology.orchestra.run.smithi084.stdout:RedHatEnterprise 2022-12-01T09:44:24.501 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:44:24.502 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:44:24.547 INFO:teuthology.orchestra.run.smithi191.stdout:RedHatEnterprise 2022-12-01T09:44:24.548 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:44:24.549 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:44:24.682 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/ 2022-12-01T09:44:24.682 INFO:teuthology.task.install.rpm:Package version is 18.0.0-676.gf6922ca2 2022-12-01T09:44:24.723 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/ 2022-12-01T09:44:24.723 INFO:teuthology.task.install.rpm:Package version is 18.0.0-676.gf6922ca2 2022-12-01T09:44:24.811 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-12-01T09:44:24.812 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:44:24.812 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-12-01T09:44:24.839 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-12-01T09:44:24.840 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f6922ca21f63dad2b198f92826f75d268f12fc97/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-12-01T09:44:24.851 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-khiremat-47893-test-mgr-finisher-thread-5/f6922ca21f63dad2b198f92826f75d268f12fc97/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-12-01T09:44:24.852 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:44:24.852 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-12-01T09:44:24.880 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-12-01T09:44:24.880 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f6922ca21f63dad2b198f92826f75d268f12fc97/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-12-01T09:44:24.908 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-12-01T09:44:24.948 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-12-01T09:44:24.985 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-12-01T09:44:25.026 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-12-01T09:44:25.052 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2022-12-01T09:44:25.053 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2022-12-01T09:44:25.091 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2022-12-01T09:44:25.093 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2022-12-01T09:44:25.424 INFO:teuthology.orchestra.run.smithi084.stdout:Updating Subscription Management repositories. 2022-12-01T09:44:25.462 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-12-01T09:44:25.668 INFO:teuthology.orchestra.run.smithi084.stdout:54 files removed 2022-12-01T09:44:25.682 INFO:teuthology.orchestra.run.smithi191.stdout:54 files removed 2022-12-01T09:44:25.715 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-12-01T09:44:25.731 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-12-01T09:44:26.044 INFO:teuthology.orchestra.run.smithi084.stdout:Updating Subscription Management repositories. 2022-12-01T09:44:26.060 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2022-12-01T09:44:26.645 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 286 kB/s | 81 kB 00:00 2022-12-01T09:44:26.675 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 273 kB/s | 81 kB 00:00 2022-12-01T09:44:26.962 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2022-12-01T09:44:26.968 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2022-12-01T09:44:27.260 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 5.6 kB/s | 1.5 kB 00:00 2022-12-01T09:44:27.279 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 5.1 kB/s | 1.5 kB 00:00 2022-12-01T09:44:28.672 INFO:teuthology.orchestra.run.smithi084.stdout:Copr repo for python3-asyncssh owned by ceph 2.5 kB/s | 3.5 kB 00:01 2022-12-01T09:44:28.821 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 2.2 kB/s | 3.5 kB 00:01 2022-12-01T09:44:31.922 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 13 MB 00:03 2022-12-01T09:44:33.694 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 13 MB 00:04 2022-12-01T09:44:34.914 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 443 kB/s | 24 kB 00:00 2022-12-01T09:44:35.724 INFO:teuthology.orchestra.run.smithi084.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 71 MB/s | 53 MB 00:00 2022-12-01T09:44:36.694 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 600 kB/s | 24 kB 00:00 2022-12-01T09:44:37.513 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 70 MB/s | 53 MB 00:00 2022-12-01T09:44:43.833 INFO:teuthology.orchestra.run.smithi084.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 62 MB/s | 48 MB 00:00 2022-12-01T09:44:45.497 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 63 MB/s | 48 MB 00:00 2022-12-01T09:44:55.004 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-12-01T09:44:55.005 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-12-01T09:44:55.020 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-12-01T09:44:55.021 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-12-01T09:44:55.022 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-12-01T09:44:55.023 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-12-01T09:44:55.025 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-12-01T09:44:55.028 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-12-01T09:44:55.029 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-12-01T09:44:55.163 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2022-12-01T09:44:55.176 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================================================== 2022-12-01T09:44:55.176 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2022-12-01T09:44:55.176 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================================================== 2022-12-01T09:44:55.176 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2022-12-01T09:44:55.177 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-12-01T09:44:55.177 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-12-01T09:44:55.177 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 6.4 k 2022-12-01T09:44:55.177 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 5.2 M 2022-12-01T09:44:55.178 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 845 k 2022-12-01T09:44:55.178 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 165 k 2022-12-01T09:44:55.178 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 1.5 M 2022-12-01T09:44:55.178 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 118 k 2022-12-01T09:44:55.179 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 1.7 M 2022-12-01T09:44:55.179 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 7.4 M 2022-12-01T09:44:55.179 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 47 k 2022-12-01T09:44:55.179 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 18 M 2022-12-01T09:44:55.180 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 55 M 2022-12-01T09:44:55.180 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 263 k 2022-12-01T09:44:55.180 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 81 k 2022-12-01T09:44:55.180 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 257 k 2022-12-01T09:44:55.181 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 15 k 2022-12-01T09:44:55.181 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-12-01T09:44:55.181 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-12-01T09:44:55.181 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-12-01T09:44:55.182 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 30 k 2022-12-01T09:44:55.182 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 768 k 2022-12-01T09:44:55.182 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 139 k 2022-12-01T09:44:55.182 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-12-01T09:44:55.183 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-12-01T09:44:55.184 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-12-01T09:44:55.184 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-12-01T09:44:55.184 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-12-01T09:44:55.184 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-12-01T09:44:55.185 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 190 k 2022-12-01T09:44:55.185 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 370 k 2022-12-01T09:44:55.185 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 364 k 2022-12-01T09:44:55.185 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 107 k 2022-12-01T09:44:55.186 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 94 k 2022-12-01T09:44:55.186 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.5 M 2022-12-01T09:44:55.186 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 177 k 2022-12-01T09:44:55.186 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-12-01T09:44:55.187 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2022-12-01T09:44:55.187 INFO:teuthology.orchestra.run.smithi084.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-12-01T09:44:55.187 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.6 M 2022-12-01T09:44:55.187 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.6 M 2022-12-01T09:44:55.188 INFO:teuthology.orchestra.run.smithi084.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-12-01T09:44:55.188 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-12-01T09:44:55.188 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2022-12-01T09:44:55.188 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-12-01T09:44:55.188 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 24 M 2022-12-01T09:44:55.189 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 23 k 2022-12-01T09:44:55.189 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 2.3 M 2022-12-01T09:44:55.189 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 234 k 2022-12-01T09:44:55.190 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.9 M 2022-12-01T09:44:55.190 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 17 M 2022-12-01T09:44:55.190 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 14 k 2022-12-01T09:44:55.190 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 24 k 2022-12-01T09:44:55.190 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-12-01T09:44:55.191 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-12-01T09:44:55.191 INFO:teuthology.orchestra.run.smithi084.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-12-01T09:44:55.191 INFO:teuthology.orchestra.run.smithi084.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-12-01T09:44:55.191 INFO:teuthology.orchestra.run.smithi084.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-12-01T09:44:55.192 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 137 k 2022-12-01T09:44:55.192 INFO:teuthology.orchestra.run.smithi084.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-12-01T09:44:55.192 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-12-01T09:44:55.192 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-12-01T09:44:55.192 INFO:teuthology.orchestra.run.smithi084.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-12-01T09:44:55.193 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-12-01T09:44:55.193 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-12-01T09:44:55.193 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 474 k 2022-12-01T09:44:55.193 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-12-01T09:44:55.194 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 6.5 M 2022-12-01T09:44:55.194 INFO:teuthology.orchestra.run.smithi084.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-12-01T09:44:55.194 INFO:teuthology.orchestra.run.smithi084.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-12-01T09:44:55.194 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-12-01T09:44:55.195 INFO:teuthology.orchestra.run.smithi084.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-12-01T09:44:55.195 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-12-01T09:44:55.195 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-12-01T09:44:55.195 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-12-01T09:44:55.195 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-12-01T09:44:55.196 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-12-01T09:44:55.196 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-12-01T09:44:55.196 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-12-01T09:44:55.196 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-12-01T09:44:55.197 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-12-01T09:44:55.197 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-12-01T09:44:55.197 INFO:teuthology.orchestra.run.smithi084.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-12-01T09:44:55.197 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-12-01T09:44:55.198 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 45 k 2022-12-01T09:44:55.198 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 109 k 2022-12-01T09:44:55.198 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-12-01T09:44:55.198 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-12-01T09:44:55.198 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-12-01T09:44:55.199 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-12-01T09:44:55.199 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-12-01T09:44:55.199 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-12-01T09:44:55.199 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-12-01T09:44:55.200 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-12-01T09:44:55.200 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-12-01T09:44:55.200 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-12-01T09:44:55.200 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-12-01T09:44:55.200 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-12-01T09:44:55.201 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-12-01T09:44:55.201 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-12-01T09:44:55.201 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-12-01T09:44:55.201 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-12-01T09:44:55.202 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-12-01T09:44:55.202 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-12-01T09:44:55.202 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-12-01T09:44:55.202 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-12-01T09:44:55.203 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-12-01T09:44:55.203 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-12-01T09:44:55.203 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-12-01T09:44:55.203 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-12-01T09:44:55.204 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-12-01T09:44:55.204 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-12-01T09:44:55.204 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-12-01T09:44:55.204 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-12-01T09:44:55.204 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-12-01T09:44:55.205 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-12-01T09:44:55.205 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-12-01T09:44:55.205 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-12-01T09:44:55.205 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-12-01T09:44:55.206 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-12-01T09:44:55.206 INFO:teuthology.orchestra.run.smithi084.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-12-01T09:44:55.206 INFO:teuthology.orchestra.run.smithi084.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-12-01T09:44:55.206 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-12-01T09:44:55.206 INFO:teuthology.orchestra.run.smithi084.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-12-01T09:44:55.207 INFO:teuthology.orchestra.run.smithi084.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-12-01T09:44:55.207 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-12-01T09:44:55.207 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-12-01T09:44:55.207 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-12-01T09:44:55.208 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-12-01T09:44:55.208 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-12-01T09:44:55.208 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-12-01T09:44:55.208 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-12-01T09:44:55.209 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-12-01T09:44:55.209 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-12-01T09:44:55.209 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-12-01T09:44:55.209 INFO:teuthology.orchestra.run.smithi084.stdout:Enabling module streams: 2022-12-01T09:44:55.209 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql 10 2022-12-01T09:44:55.210 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:44:55.210 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2022-12-01T09:44:55.210 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================================================== 2022-12-01T09:44:55.210 INFO:teuthology.orchestra.run.smithi084.stdout:Install 128 Packages 2022-12-01T09:44:55.211 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 5 Packages 2022-12-01T09:44:55.211 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:44:55.212 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 219 M 2022-12-01T09:44:55.212 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2022-12-01T09:44:55.323 INFO:teuthology.orchestra.run.smithi084.stdout:(1/133): ceph-18.0.0-676.gf6922ca2.el8.x86_64.r 49 kB/s | 6.4 kB 00:00 2022-12-01T09:44:55.907 INFO:teuthology.orchestra.run.smithi084.stdout:(2/133): ceph-fuse-18.0.0-676.gf6922ca2.el8.x86 1.4 MB/s | 845 kB 00:00 2022-12-01T09:44:56.040 INFO:teuthology.orchestra.run.smithi084.stdout:(3/133): ceph-immutable-object-cache-18.0.0-676 1.2 MB/s | 165 kB 00:00 2022-12-01T09:44:56.500 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-12-01T09:44:56.501 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-12-01T09:44:56.507 INFO:teuthology.orchestra.run.smithi084.stdout:(4/133): ceph-base-18.0.0-676.gf6922ca2.el8.x86 3.9 MB/s | 5.2 MB 00:01 2022-12-01T09:44:56.516 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-12-01T09:44:56.517 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-12-01T09:44:56.518 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-12-01T09:44:56.519 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-12-01T09:44:56.521 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-12-01T09:44:56.524 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-12-01T09:44:56.526 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-12-01T09:44:56.658 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2022-12-01T09:44:56.670 INFO:teuthology.orchestra.run.smithi191.stdout:=========================================================================================================================== 2022-12-01T09:44:56.671 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2022-12-01T09:44:56.671 INFO:teuthology.orchestra.run.smithi191.stdout:=========================================================================================================================== 2022-12-01T09:44:56.671 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2022-12-01T09:44:56.672 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-12-01T09:44:56.672 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-12-01T09:44:56.672 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 6.4 k 2022-12-01T09:44:56.672 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 5.2 M 2022-12-01T09:44:56.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 845 k 2022-12-01T09:44:56.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 165 k 2022-12-01T09:44:56.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 1.5 M 2022-12-01T09:44:56.673 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 118 k 2022-12-01T09:44:56.674 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 1.7 M 2022-12-01T09:44:56.674 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 7.4 M 2022-12-01T09:44:56.674 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 47 k 2022-12-01T09:44:56.674 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 18 M 2022-12-01T09:44:56.675 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 55 M 2022-12-01T09:44:56.675 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 263 k 2022-12-01T09:44:56.675 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 81 k 2022-12-01T09:44:56.675 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 257 k 2022-12-01T09:44:56.676 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 15 k 2022-12-01T09:44:56.676 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-12-01T09:44:56.676 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-12-01T09:44:56.676 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-12-01T09:44:56.677 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 30 k 2022-12-01T09:44:56.677 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 768 k 2022-12-01T09:44:56.677 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 139 k 2022-12-01T09:44:56.677 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-12-01T09:44:56.678 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-12-01T09:44:56.678 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-12-01T09:44:56.678 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-12-01T09:44:56.678 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-12-01T09:44:56.679 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-12-01T09:44:56.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 190 k 2022-12-01T09:44:56.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 370 k 2022-12-01T09:44:56.679 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 364 k 2022-12-01T09:44:56.680 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 107 k 2022-12-01T09:44:56.680 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 94 k 2022-12-01T09:44:56.680 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.5 M 2022-12-01T09:44:56.680 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 177 k 2022-12-01T09:44:56.680 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-12-01T09:44:56.681 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2022-12-01T09:44:56.681 INFO:teuthology.orchestra.run.smithi191.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-12-01T09:44:56.681 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.6 M 2022-12-01T09:44:56.681 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.6 M 2022-12-01T09:44:56.682 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-12-01T09:44:56.682 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-12-01T09:44:56.682 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2022-12-01T09:44:56.682 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-12-01T09:44:56.683 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 24 M 2022-12-01T09:44:56.683 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 23 k 2022-12-01T09:44:56.683 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 2.3 M 2022-12-01T09:44:56.683 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 234 k 2022-12-01T09:44:56.683 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 3.9 M 2022-12-01T09:44:56.684 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 17 M 2022-12-01T09:44:56.684 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:18.0.0-676.gf6922ca2.el8 ceph-noarch 14 k 2022-12-01T09:44:56.684 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 24 k 2022-12-01T09:44:56.684 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-12-01T09:44:56.685 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-12-01T09:44:56.685 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-12-01T09:44:56.685 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-12-01T09:44:56.685 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-12-01T09:44:56.686 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 137 k 2022-12-01T09:44:56.686 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-12-01T09:44:56.686 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-12-01T09:44:56.686 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-12-01T09:44:56.686 INFO:teuthology.orchestra.run.smithi191.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-12-01T09:44:56.687 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-12-01T09:44:56.687 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-12-01T09:44:56.687 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 474 k 2022-12-01T09:44:56.687 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-12-01T09:44:56.688 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 6.5 M 2022-12-01T09:44:56.688 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-12-01T09:44:56.688 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-12-01T09:44:56.688 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-12-01T09:44:56.689 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-12-01T09:44:56.689 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-12-01T09:44:56.689 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-12-01T09:44:56.689 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-12-01T09:44:56.690 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-12-01T09:44:56.690 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-12-01T09:44:56.690 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-12-01T09:44:56.690 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-12-01T09:44:56.690 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-12-01T09:44:56.691 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-12-01T09:44:56.691 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-12-01T09:44:56.691 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-12-01T09:44:56.691 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-12-01T09:44:56.692 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 45 k 2022-12-01T09:44:56.692 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:18.0.0-676.gf6922ca2.el8 ceph 109 k 2022-12-01T09:44:56.692 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-12-01T09:44:56.692 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-12-01T09:44:56.692 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-12-01T09:44:56.693 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-12-01T09:44:56.693 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-12-01T09:44:56.693 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-12-01T09:44:56.693 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-12-01T09:44:56.694 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-12-01T09:44:56.694 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-12-01T09:44:56.694 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-12-01T09:44:56.694 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-12-01T09:44:56.694 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-12-01T09:44:56.695 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-12-01T09:44:56.695 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-12-01T09:44:56.695 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-12-01T09:44:56.696 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-12-01T09:44:56.696 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-12-01T09:44:56.696 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-12-01T09:44:56.696 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-12-01T09:44:56.696 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-12-01T09:44:56.697 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-12-01T09:44:56.697 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-12-01T09:44:56.697 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-12-01T09:44:56.697 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-12-01T09:44:56.698 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-12-01T09:44:56.698 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-12-01T09:44:56.698 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-12-01T09:44:56.698 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-12-01T09:44:56.699 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-12-01T09:44:56.699 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-12-01T09:44:56.699 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-12-01T09:44:56.699 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-12-01T09:44:56.700 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-12-01T09:44:56.700 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-12-01T09:44:56.700 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-12-01T09:44:56.700 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-12-01T09:44:56.700 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-12-01T09:44:56.701 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-12-01T09:44:56.701 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-12-01T09:44:56.701 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-12-01T09:44:56.701 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-12-01T09:44:56.702 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-12-01T09:44:56.702 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-12-01T09:44:56.702 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-12-01T09:44:56.702 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-12-01T09:44:56.703 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-12-01T09:44:56.703 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-12-01T09:44:56.703 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-12-01T09:44:56.703 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-12-01T09:44:56.704 INFO:teuthology.orchestra.run.smithi191.stdout:Enabling module streams: 2022-12-01T09:44:56.704 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql 10 2022-12-01T09:44:56.704 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:44:56.704 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2022-12-01T09:44:56.704 INFO:teuthology.orchestra.run.smithi191.stdout:=========================================================================================================================== 2022-12-01T09:44:56.705 INFO:teuthology.orchestra.run.smithi191.stdout:Install 128 Packages 2022-12-01T09:44:56.705 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 5 Packages 2022-12-01T09:44:56.705 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:44:56.706 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 219 M 2022-12-01T09:44:56.707 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2022-12-01T09:44:56.741 INFO:teuthology.orchestra.run.smithi084.stdout:(5/133): ceph-mgr-18.0.0-676.gf6922ca2.el8.x86_ 6.6 MB/s | 1.5 MB 00:00 2022-12-01T09:44:56.888 INFO:teuthology.orchestra.run.smithi191.stdout:(1/133): ceph-18.0.0-676.gf6922ca2.el8.x86_64.r 32 kB/s | 6.4 kB 00:00 2022-12-01T09:44:57.206 INFO:teuthology.orchestra.run.smithi191.stdout:(2/133): ceph-fuse-18.0.0-676.gf6922ca2.el8.x86 2.6 MB/s | 845 kB 00:00 2022-12-01T09:44:57.235 INFO:teuthology.orchestra.run.smithi084.stdout:(6/133): ceph-mds-18.0.0-676.gf6922ca2.el8.x86_ 1.9 MB/s | 2.3 MB 00:01 2022-12-01T09:44:57.256 INFO:teuthology.orchestra.run.smithi191.stdout:(3/133): ceph-immutable-object-cache-18.0.0-676 3.2 MB/s | 165 kB 00:00 2022-12-01T09:44:57.285 INFO:teuthology.orchestra.run.smithi084.stdout:(7/133): ceph-mon-18.0.0-676.gf6922ca2.el8.x86_ 7.1 MB/s | 3.9 MB 00:00 2022-12-01T09:44:57.589 INFO:teuthology.orchestra.run.smithi191.stdout:(4/133): ceph-mds-18.0.0-676.gf6922ca2.el8.x86_ 6.8 MB/s | 2.3 MB 00:00 2022-12-01T09:44:57.831 INFO:teuthology.orchestra.run.smithi191.stdout:(5/133): ceph-mgr-18.0.0-676.gf6922ca2.el8.x86_ 6.3 MB/s | 1.5 MB 00:00 2022-12-01T09:44:58.365 INFO:teuthology.orchestra.run.smithi191.stdout:(6/133): ceph-mon-18.0.0-676.gf6922ca2.el8.x86_ 7.2 MB/s | 3.9 MB 00:00 2022-12-01T09:44:58.685 INFO:teuthology.orchestra.run.smithi084.stdout:(8/133): ceph-common-18.0.0-676.gf6922ca2.el8.x 7.0 MB/s | 24 MB 00:03 2022-12-01T09:44:58.727 INFO:teuthology.orchestra.run.smithi084.stdout:(9/133): ceph-selinux-18.0.0-676.gf6922ca2.el8. 588 kB/s | 24 kB 00:00 2022-12-01T09:44:59.549 INFO:teuthology.orchestra.run.smithi191.stdout:(7/133): ceph-base-18.0.0-676.gf6922ca2.el8.x86 1.8 MB/s | 5.2 MB 00:02 2022-12-01T09:45:00.360 INFO:teuthology.orchestra.run.smithi084.stdout:(10/133): ceph-radosgw-18.0.0-676.gf6922ca2.el8 5.9 MB/s | 18 MB 00:03 2022-12-01T09:45:00.390 INFO:teuthology.orchestra.run.smithi191.stdout:(8/133): ceph-osd-18.0.0-676.gf6922ca2.el8.x86_ 8.3 MB/s | 17 MB 00:02 2022-12-01T09:45:00.427 INFO:teuthology.orchestra.run.smithi084.stdout:(11/133): cephfs-mirror-18.0.0-676.gf6922ca2.el 3.8 MB/s | 257 kB 00:00 2022-12-01T09:45:00.441 INFO:teuthology.orchestra.run.smithi191.stdout:(9/133): ceph-selinux-18.0.0-676.gf6922ca2.el8. 490 kB/s | 24 kB 00:00 2022-12-01T09:45:00.494 INFO:teuthology.orchestra.run.smithi084.stdout:(12/133): libcephfs-devel-18.0.0-676.gf6922ca2. 458 kB/s | 30 kB 00:00 2022-12-01T09:45:00.678 INFO:teuthology.orchestra.run.smithi084.stdout:(13/133): libcephfs2-18.0.0-676.gf6922ca2.el8.x 4.1 MB/s | 768 kB 00:00 2022-12-01T09:45:00.730 INFO:teuthology.orchestra.run.smithi084.stdout:(14/133): libcephsqlite-18.0.0-676.gf6922ca2.el 2.7 MB/s | 137 kB 00:00 2022-12-01T09:45:00.820 INFO:teuthology.orchestra.run.smithi084.stdout:(15/133): librados-devel-18.0.0-676.gf6922ca2.e 1.5 MB/s | 139 kB 00:00 2022-12-01T09:45:00.962 INFO:teuthology.orchestra.run.smithi084.stdout:(16/133): libradosstriper1-18.0.0-676.gf6922ca2 3.3 MB/s | 474 kB 00:00 2022-12-01T09:45:02.212 INFO:teuthology.orchestra.run.smithi084.stdout:(17/133): librgw2-18.0.0-676.gf6922ca2.el8.x86_ 5.2 MB/s | 6.5 MB 00:01 2022-12-01T09:45:02.270 INFO:teuthology.orchestra.run.smithi084.stdout:(18/133): python3-ceph-argparse-18.0.0-676.gf69 767 kB/s | 45 kB 00:00 2022-12-01T09:45:02.338 INFO:teuthology.orchestra.run.smithi084.stdout:(19/133): python3-ceph-common-18.0.0-676.gf6922 1.6 MB/s | 109 kB 00:00 2022-12-01T09:45:02.404 INFO:teuthology.orchestra.run.smithi084.stdout:(20/133): python3-cephfs-18.0.0-676.gf6922ca2.e 2.8 MB/s | 190 kB 00:00 2022-12-01T09:45:02.521 INFO:teuthology.orchestra.run.smithi084.stdout:(21/133): python3-rados-18.0.0-676.gf6922ca2.el 3.1 MB/s | 370 kB 00:00 2022-12-01T09:45:02.638 INFO:teuthology.orchestra.run.smithi084.stdout:(22/133): python3-rbd-18.0.0-676.gf6922ca2.el8. 3.0 MB/s | 364 kB 00:00 2022-12-01T09:45:02.705 INFO:teuthology.orchestra.run.smithi084.stdout:(23/133): python3-rgw-18.0.0-676.gf6922ca2.el8. 1.6 MB/s | 107 kB 00:00 2022-12-01T09:45:02.764 INFO:teuthology.orchestra.run.smithi084.stdout:(24/133): rbd-fuse-18.0.0-676.gf6922ca2.el8.x86 1.6 MB/s | 94 kB 00:00 2022-12-01T09:45:03.491 INFO:teuthology.orchestra.run.smithi084.stdout:(25/133): rbd-mirror-18.0.0-676.gf6922ca2.el8.x 4.8 MB/s | 3.5 MB 00:00 2022-12-01T09:45:03.564 INFO:teuthology.orchestra.run.smithi084.stdout:(26/133): rbd-nbd-18.0.0-676.gf6922ca2.el8.x86_ 2.4 MB/s | 177 kB 00:00 2022-12-01T09:45:03.631 INFO:teuthology.orchestra.run.smithi084.stdout:(27/133): ceph-grafana-dashboards-18.0.0-676.gf 344 kB/s | 23 kB 00:00 2022-12-01T09:45:03.723 INFO:teuthology.orchestra.run.smithi084.stdout:(28/133): ceph-mgr-cephadm-18.0.0-676.gf6922ca2 1.3 MB/s | 118 kB 00:00 2022-12-01T09:45:04.132 INFO:teuthology.orchestra.run.smithi084.stdout:(29/133): ceph-mgr-dashboard-18.0.0-676.gf6922c 4.0 MB/s | 1.7 MB 00:00 2022-12-01T09:45:04.765 INFO:teuthology.orchestra.run.smithi084.stdout:(30/133): ceph-osd-18.0.0-676.gf6922ca2.el8.x86 2.2 MB/s | 17 MB 00:07 2022-12-01T09:45:04.874 INFO:teuthology.orchestra.run.smithi084.stdout:(31/133): ceph-mgr-modules-core-18.0.0-676.gf69 2.1 MB/s | 234 kB 00:00 2022-12-01T09:45:04.924 INFO:teuthology.orchestra.run.smithi084.stdout:(32/133): ceph-mgr-rook-18.0.0-676.gf6922ca2.el 941 kB/s | 47 kB 00:00 2022-12-01T09:45:04.983 INFO:teuthology.orchestra.run.smithi084.stdout:(33/133): ceph-prometheus-alerts-18.0.0-676.gf6 247 kB/s | 14 kB 00:00 2022-12-01T09:45:05.133 INFO:teuthology.orchestra.run.smithi084.stdout:(34/133): ceph-volume-18.0.0-676.gf6922ca2.el8. 1.7 MB/s | 263 kB 00:00 2022-12-01T09:45:05.208 INFO:teuthology.orchestra.run.smithi084.stdout:(35/133): cephadm-18.0.0-676.gf6922ca2.el8.noar 1.1 MB/s | 81 kB 00:00 2022-12-01T09:45:05.259 INFO:teuthology.orchestra.run.smithi084.stdout:(36/133): cephfs-top-18.0.0-676.gf6922ca2.el8.n 308 kB/s | 15 kB 00:00 2022-12-01T09:45:05.343 INFO:teuthology.orchestra.run.smithi084.stdout:(37/133): ceph-mgr-diskprediction-local-18.0.0- 6.1 MB/s | 7.4 MB 00:01 2022-12-01T09:45:05.476 INFO:teuthology.orchestra.run.smithi084.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 442 kB/s | 59 kB 00:00 2022-12-01T09:45:05.560 INFO:teuthology.orchestra.run.smithi084.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 899 kB/s | 75 kB 00:00 2022-12-01T09:45:05.565 INFO:teuthology.orchestra.run.smithi191.stdout:(10/133): ceph-radosgw-18.0.0-676.gf6922ca2.el8 3.0 MB/s | 18 MB 00:06 2022-12-01T09:45:05.585 INFO:teuthology.orchestra.run.smithi084.stdout:(40/133): gperftools-libs-2.7-9.el8.x86_64.rpm 940 kB/s | 306 kB 00:00 2022-12-01T09:45:05.627 INFO:teuthology.orchestra.run.smithi084.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2022-12-01T09:45:05.665 INFO:teuthology.orchestra.run.smithi191.stdout:(11/133): cephfs-mirror-18.0.0-676.gf6922ca2.el 2.5 MB/s | 257 kB 00:00 2022-12-01T09:45:05.715 INFO:teuthology.orchestra.run.smithi191.stdout:(12/133): libcephfs-devel-18.0.0-676.gf6922ca2. 612 kB/s | 30 kB 00:00 2022-12-01T09:45:05.736 INFO:teuthology.orchestra.run.smithi084.stdout:(42/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2022-12-01T09:45:05.786 INFO:teuthology.orchestra.run.smithi084.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 661 kB/s | 33 kB 00:00 2022-12-01T09:45:05.820 INFO:teuthology.orchestra.run.smithi084.stdout:(44/133): python3-asyncssh-2.7.0-2.el8.noarch.r 1.8 MB/s | 476 kB 00:00 2022-12-01T09:45:05.871 INFO:teuthology.orchestra.run.smithi084.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2022-12-01T09:45:05.899 INFO:teuthology.orchestra.run.smithi191.stdout:(13/133): libcephfs2-18.0.0-676.gf6922ca2.el8.x 4.1 MB/s | 768 kB 00:00 2022-12-01T09:45:05.953 INFO:teuthology.orchestra.run.smithi084.stdout:(46/133): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2022-12-01T09:45:06.000 INFO:teuthology.orchestra.run.smithi191.stdout:(14/133): libcephsqlite-18.0.0-676.gf6922ca2.el 1.3 MB/s | 137 kB 00:00 2022-12-01T09:45:06.028 INFO:teuthology.orchestra.run.smithi084.stdout:(47/133): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2022-12-01T09:45:06.075 INFO:teuthology.orchestra.run.smithi191.stdout:(15/133): librados-devel-18.0.0-676.gf6922ca2.e 1.8 MB/s | 139 kB 00:00 2022-12-01T09:45:06.079 INFO:teuthology.orchestra.run.smithi084.stdout:(48/133): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2022-12-01T09:45:06.104 INFO:teuthology.orchestra.run.smithi084.stdout:(49/133): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2022-12-01T09:45:06.130 INFO:teuthology.orchestra.run.smithi084.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2022-12-01T09:45:06.213 INFO:teuthology.orchestra.run.smithi084.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 583 kB/s | 49 kB 00:00 2022-12-01T09:45:06.226 INFO:teuthology.orchestra.run.smithi191.stdout:(16/133): libradosstriper1-18.0.0-676.gf6922ca2 3.1 MB/s | 474 kB 00:00 2022-12-01T09:45:06.264 INFO:teuthology.orchestra.run.smithi084.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2022-12-01T09:45:06.313 INFO:teuthology.orchestra.run.smithi084.stdout:(53/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2022-12-01T09:45:06.456 INFO:teuthology.orchestra.run.smithi084.stdout:(54/133): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2022-12-01T09:45:06.534 INFO:teuthology.orchestra.run.smithi084.stdout:(55/133): python3-portend-2.6-1.el8.noarch.rpm 198 kB/s | 16 kB 00:00 2022-12-01T09:45:06.590 INFO:teuthology.orchestra.run.smithi084.stdout:(56/133): python3-repoze-lru-0.7-6.el8.noarch.r 616 kB/s | 34 kB 00:00 2022-12-01T09:45:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:(57/133): python3-kubernetes-11.0.0-6.el8.noarc 2.1 MB/s | 1.2 MB 00:00 2022-12-01T09:45:06.711 INFO:teuthology.orchestra.run.smithi084.stdout:(58/133): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2022-12-01T09:45:06.732 INFO:teuthology.orchestra.run.smithi084.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 819 kB/s | 61 kB 00:00 2022-12-01T09:45:06.757 INFO:teuthology.orchestra.run.smithi084.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2022-12-01T09:45:06.783 INFO:teuthology.orchestra.run.smithi084.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 475 kB/s | 24 kB 00:00 2022-12-01T09:45:06.808 INFO:teuthology.orchestra.run.smithi084.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 579 kB/s | 29 kB 00:00 2022-12-01T09:45:06.833 INFO:teuthology.orchestra.run.smithi084.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2022-12-01T09:45:06.917 INFO:teuthology.orchestra.run.smithi191.stdout:(17/133): ceph-common-18.0.0-676.gf6922ca2.el8. 2.4 MB/s | 24 MB 00:10 2022-12-01T09:45:07.434 INFO:teuthology.orchestra.run.smithi191.stdout:(18/133): ceph-test-18.0.0-676.gf6922ca2.el8.x8 7.8 MB/s | 55 MB 00:06 2022-12-01T09:45:07.460 INFO:teuthology.orchestra.run.smithi191.stdout:(19/133): python3-ceph-argparse-18.0.0-676.gf69 82 kB/s | 45 kB 00:00 2022-12-01T09:45:07.510 INFO:teuthology.orchestra.run.smithi191.stdout:(20/133): python3-ceph-common-18.0.0-676.gf6922 1.4 MB/s | 109 kB 00:00 2022-12-01T09:45:07.577 INFO:teuthology.orchestra.run.smithi191.stdout:(21/133): python3-cephfs-18.0.0-676.gf6922ca2.e 1.6 MB/s | 190 kB 00:00 2022-12-01T09:45:07.602 INFO:teuthology.orchestra.run.smithi191.stdout:(22/133): python3-rados-18.0.0-676.gf6922ca2.el 3.9 MB/s | 370 kB 00:00 2022-12-01T09:45:07.625 INFO:teuthology.orchestra.run.smithi084.stdout:(64/133): ceph-test-18.0.0-676.gf6922ca2.el8.x8 6.2 MB/s | 55 MB 00:08 2022-12-01T09:45:07.651 INFO:teuthology.orchestra.run.smithi084.stdout:(65/133): python3-waitress-1.2.1-2.el8.1.noarch 286 kB/s | 241 kB 00:00 2022-12-01T09:45:07.652 INFO:teuthology.orchestra.run.smithi191.stdout:(23/133): python3-rgw-18.0.0-676.gf6922ca2.el8. 2.1 MB/s | 107 kB 00:00 2022-12-01T09:45:07.676 INFO:teuthology.orchestra.run.smithi084.stdout:(66/133): python3-webob-1.8.5-1.el8.1.noarch.rp 298 kB/s | 251 kB 00:00 2022-12-01T09:45:07.690 INFO:teuthology.orchestra.run.smithi191.stdout:(24/133): python3-rbd-18.0.0-676.gf6922ca2.el8. 3.3 MB/s | 364 kB 00:00 2022-12-01T09:45:07.712 INFO:teuthology.orchestra.run.smithi191.stdout:(25/133): rbd-fuse-18.0.0-676.gf6922ca2.el8.x86 1.6 MB/s | 94 kB 00:00 2022-12-01T09:45:07.726 INFO:teuthology.orchestra.run.smithi084.stdout:(67/133): python3-zc-lockfile-2.0-2.el8.noarch. 460 kB/s | 23 kB 00:00 2022-12-01T09:45:07.760 INFO:teuthology.orchestra.run.smithi084.stdout:(68/133): python3-webtest-2.0.33-1.el8.noarch.r 786 kB/s | 85 kB 00:00 2022-12-01T09:45:07.763 INFO:teuthology.orchestra.run.smithi191.stdout:(26/133): rbd-nbd-18.0.0-676.gf6922ca2.el8.x86_ 3.5 MB/s | 177 kB 00:00 2022-12-01T09:45:07.785 INFO:teuthology.orchestra.run.smithi084.stdout:(69/133): python3-websocket-client-0.56.0-5.el8 385 kB/s | 61 kB 00:00 2022-12-01T09:45:07.816 INFO:teuthology.orchestra.run.smithi191.stdout:(27/133): ceph-grafana-dashboards-18.0.0-676.gf 457 kB/s | 23 kB 00:00 2022-12-01T09:45:07.816 INFO:teuthology.orchestra.run.smithi084.stdout:(70/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2022-12-01T09:45:07.862 INFO:teuthology.orchestra.run.smithi191.stdout:(28/133): ceph-mgr-cephadm-18.0.0-676.gf6922ca2 2.3 MB/s | 118 kB 00:00 2022-12-01T09:45:07.869 INFO:teuthology.orchestra.run.smithi084.stdout:(71/133): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2022-12-01T09:45:07.895 INFO:teuthology.orchestra.run.smithi084.stdout:(72/133): fuse-2.9.7-15.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2022-12-01T09:45:07.936 INFO:teuthology.orchestra.run.smithi084.stdout:(73/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2022-12-01T09:45:07.979 INFO:teuthology.orchestra.run.smithi084.stdout:(74/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2022-12-01T09:45:08.028 INFO:teuthology.orchestra.run.smithi084.stdout:(75/133): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2022-12-01T09:45:08.045 INFO:teuthology.orchestra.run.smithi084.stdout:(76/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 277 kB/s | 18 kB 00:00 2022-12-01T09:45:08.062 INFO:teuthology.orchestra.run.smithi191.stdout:(29/133): ceph-mgr-dashboard-18.0.0-676.gf6922c 8.3 MB/s | 1.7 MB 00:00 2022-12-01T09:45:08.079 INFO:teuthology.orchestra.run.smithi084.stdout:(77/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 936 kB/s | 47 kB 00:00 2022-12-01T09:45:08.112 INFO:teuthology.orchestra.run.smithi084.stdout:(78/133): python3-cryptography-3.2.1-5.el8.x86_ 8.2 MB/s | 559 kB 00:00 2022-12-01T09:45:08.146 INFO:teuthology.orchestra.run.smithi084.stdout:(79/133): python3-oauthlib-2.1.0-1.el8.noarch.r 2.3 MB/s | 155 kB 00:00 2022-12-01T09:45:08.172 INFO:teuthology.orchestra.run.smithi084.stdout:(80/133): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2022-12-01T09:45:08.179 INFO:teuthology.orchestra.run.smithi191.stdout:(30/133): librgw2-18.0.0-676.gf6922ca2.el8.x86_ 3.3 MB/s | 6.5 MB 00:01 2022-12-01T09:45:08.230 INFO:teuthology.orchestra.run.smithi084.stdout:(81/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.5 MB/s | 560 kB 00:00 2022-12-01T09:45:08.255 INFO:teuthology.orchestra.run.smithi084.stdout:(82/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.0 MB/s | 171 kB 00:00 2022-12-01T09:45:08.271 INFO:teuthology.orchestra.run.smithi191.stdout:(31/133): ceph-mgr-modules-core-18.0.0-676.gf69 2.5 MB/s | 234 kB 00:00 2022-12-01T09:45:08.297 INFO:teuthology.orchestra.run.smithi084.stdout:(83/133): python3-jwt-1.6.1-2.el8.noarch.rpm 638 kB/s | 43 kB 00:00 2022-12-01T09:45:08.322 INFO:teuthology.orchestra.run.smithi191.stdout:(32/133): ceph-mgr-rook-18.0.0-676.gf6922ca2.el 941 kB/s | 47 kB 00:00 2022-12-01T09:45:08.347 INFO:teuthology.orchestra.run.smithi084.stdout:(84/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 627 kB/s | 58 kB 00:00 2022-12-01T09:45:08.381 INFO:teuthology.orchestra.run.smithi084.stdout:(85/133): elfutils-libelf-devel-0.186-1.el8.x86 717 kB/s | 60 kB 00:00 2022-12-01T09:45:08.388 INFO:teuthology.orchestra.run.smithi191.stdout:(33/133): ceph-prometheus-alerts-18.0.0-676.gf6 217 kB/s | 14 kB 00:00 2022-12-01T09:45:08.406 INFO:teuthology.orchestra.run.smithi084.stdout:(86/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2022-12-01T09:45:08.440 INFO:teuthology.orchestra.run.smithi084.stdout:(87/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2022-12-01T09:45:08.465 INFO:teuthology.orchestra.run.smithi084.stdout:(88/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.7 MB/s | 98 kB 00:00 2022-12-01T09:45:08.497 INFO:teuthology.orchestra.run.smithi191.stdout:(34/133): rbd-mirror-18.0.0-676.gf6922ca2.el8.x 4.3 MB/s | 3.5 MB 00:00 2022-12-01T09:45:08.507 INFO:teuthology.orchestra.run.smithi084.stdout:(89/133): thrift-0.13.0-2.el8.x86_64.rpm 2.2 MB/s | 1.7 MB 00:00 2022-12-01T09:45:08.522 INFO:teuthology.orchestra.run.smithi191.stdout:(35/133): ceph-volume-18.0.0-676.gf6922ca2.el8. 1.9 MB/s | 263 kB 00:00 2022-12-01T09:45:08.542 INFO:teuthology.orchestra.run.smithi084.stdout:(90/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.2 MB/s | 229 kB 00:00 2022-12-01T09:45:08.548 INFO:teuthology.orchestra.run.smithi191.stdout:(36/133): cephadm-18.0.0-676.gf6922ca2.el8.noar 1.6 MB/s | 81 kB 00:00 2022-12-01T09:45:08.566 INFO:teuthology.orchestra.run.smithi084.stdout:(91/133): python3-requests-oauthlib-1.0.0-1.el8 428 kB/s | 43 kB 00:00 2022-12-01T09:45:08.573 INFO:teuthology.orchestra.run.smithi191.stdout:(37/133): cephfs-top-18.0.0-676.gf6922ca2.el8.n 308 kB/s | 15 kB 00:00 2022-12-01T09:45:08.608 INFO:teuthology.orchestra.run.smithi084.stdout:(92/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 719 kB/s | 72 kB 00:00 2022-12-01T09:45:08.633 INFO:teuthology.orchestra.run.smithi084.stdout:(93/133): libcom_err-devel-1.45.6-4.el8.x86_64. 419 kB/s | 39 kB 00:00 2022-12-01T09:45:08.692 INFO:teuthology.orchestra.run.smithi084.stdout:(94/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 576 kB/s | 48 kB 00:00 2022-12-01T09:45:08.715 INFO:teuthology.orchestra.run.smithi191.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 415 kB/s | 59 kB 00:00 2022-12-01T09:45:08.742 INFO:teuthology.orchestra.run.smithi084.stdout:(95/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2022-12-01T09:45:08.776 INFO:teuthology.orchestra.run.smithi084.stdout:(96/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.4 MB/s | 644 kB 00:00 2022-12-01T09:45:08.790 INFO:teuthology.orchestra.run.smithi191.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2022-12-01T09:45:08.801 INFO:teuthology.orchestra.run.smithi084.stdout:(97/133): NetworkManager-initscripts-updown-1.3 1.2 MB/s | 138 kB 00:00 2022-12-01T09:45:08.844 INFO:teuthology.orchestra.run.smithi191.stdout:(40/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2022-12-01T09:45:08.868 INFO:teuthology.orchestra.run.smithi084.stdout:(98/133): postgresql-10.21-2.module+el8.6.0+153 23 MB/s | 1.5 MB 00:00 2022-12-01T09:45:08.899 INFO:teuthology.orchestra.run.smithi191.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 805 kB/s | 44 kB 00:00 2022-12-01T09:45:08.969 INFO:teuthology.orchestra.run.smithi084.stdout:(99/133): postgresql-contrib-10.21-2.module+el8 3.5 MB/s | 808 kB 00:00 2022-12-01T09:45:09.027 INFO:teuthology.orchestra.run.smithi084.stdout:(100/133): bison-3.0.4-10.el8.x86_64.rpm 2.7 MB/s | 688 kB 00:00 2022-12-01T09:45:09.041 INFO:teuthology.orchestra.run.smithi191.stdout:(42/133): ceph-mgr-diskprediction-local-18.0.0- 7.6 MB/s | 7.4 MB 00:00 2022-12-01T09:45:09.066 INFO:teuthology.orchestra.run.smithi191.stdout:(43/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2022-12-01T09:45:09.069 INFO:teuthology.orchestra.run.smithi084.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2022-12-01T09:45:09.092 INFO:teuthology.orchestra.run.smithi191.stdout:(44/133): python3-asyncssh-2.7.0-2.el8.noarch.r 1.5 MB/s | 476 kB 00:00 2022-12-01T09:45:09.117 INFO:teuthology.orchestra.run.smithi191.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 324 kB/s | 16 kB 00:00 2022-12-01T09:45:09.128 INFO:teuthology.orchestra.run.smithi084.stdout:(102/133): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2022-12-01T09:45:09.161 INFO:teuthology.orchestra.run.smithi084.stdout:(103/133): python3-markupsafe-0.23-19.el8.x86_6 423 kB/s | 39 kB 00:00 2022-12-01T09:45:09.167 INFO:teuthology.orchestra.run.smithi191.stdout:(46/133): python3-cachetools-3.1.1-4.el8.noarch 263 kB/s | 33 kB 00:00 2022-12-01T09:45:09.186 INFO:teuthology.orchestra.run.smithi084.stdout:(104/133): python3-prettytable-0.7.2-14.el8.noa 768 kB/s | 44 kB 00:00 2022-12-01T09:45:09.234 INFO:teuthology.orchestra.run.smithi191.stdout:(47/133): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2022-12-01T09:45:09.259 INFO:teuthology.orchestra.run.smithi191.stdout:(48/133): python3-google-auth-1.1.1-10.el8.noar 884 kB/s | 81 kB 00:00 2022-12-01T09:45:09.262 INFO:teuthology.orchestra.run.smithi084.stdout:(105/133): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2022-12-01T09:45:09.285 INFO:teuthology.orchestra.run.smithi191.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2022-12-01T09:45:09.295 INFO:teuthology.orchestra.run.smithi084.stdout:(106/133): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2022-12-01T09:45:09.310 INFO:teuthology.orchestra.run.smithi191.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2022-12-01T09:45:09.345 INFO:teuthology.orchestra.run.smithi084.stdout:(107/133): python3-jinja2-2.10.1-3.el8.noarch.r 3.3 MB/s | 538 kB 00:00 2022-12-01T09:45:09.352 INFO:teuthology.orchestra.run.smithi191.stdout:(51/133): python3-cherrypy-18.4.0-1.el8.noarch. 1.6 MB/s | 384 kB 00:00 2022-12-01T09:45:09.369 INFO:teuthology.orchestra.run.smithi191.stdout:(52/133): python3-logutils-0.3.5-11.el8.noarch. 830 kB/s | 49 kB 00:00 2022-12-01T09:45:09.387 INFO:teuthology.orchestra.run.smithi084.stdout:(108/133): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2022-12-01T09:45:09.403 INFO:teuthology.orchestra.run.smithi191.stdout:(53/133): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2022-12-01T09:45:09.404 INFO:teuthology.orchestra.run.smithi084.stdout:(109/133): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2022-12-01T09:45:09.428 INFO:teuthology.orchestra.run.smithi191.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2022-12-01T09:45:09.452 INFO:teuthology.orchestra.run.smithi084.stdout:(110/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2022-12-01T09:45:09.478 INFO:teuthology.orchestra.run.smithi191.stdout:(55/133): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2022-12-01T09:45:09.488 INFO:teuthology.orchestra.run.smithi084.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2022-12-01T09:45:09.529 INFO:teuthology.orchestra.run.smithi191.stdout:(56/133): python3-repoze-lru-0.7-6.el8.noarch.r 676 kB/s | 34 kB 00:00 2022-12-01T09:45:09.555 INFO:teuthology.orchestra.run.smithi084.stdout:(112/133): openblas-threads-0.3.15-3.el8.x86_64 23 MB/s | 4.9 MB 00:00 2022-12-01T09:45:09.577 INFO:teuthology.orchestra.run.smithi191.stdout:(57/133): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2022-12-01T09:45:09.597 INFO:teuthology.orchestra.run.smithi084.stdout:(113/133): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2022-12-01T09:45:09.623 INFO:teuthology.orchestra.run.smithi084.stdout:(114/133): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2022-12-01T09:45:09.629 INFO:teuthology.orchestra.run.smithi191.stdout:(58/133): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2022-12-01T09:45:09.723 INFO:teuthology.orchestra.run.smithi084.stdout:(115/133): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2022-12-01T09:45:09.764 INFO:teuthology.orchestra.run.smithi084.stdout:(116/133): jq-1.6-3.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2022-12-01T09:45:09.779 INFO:teuthology.orchestra.run.smithi191.stdout:(59/133): python3-simplegeneric-0.8.1-17.el8.no 125 kB/s | 19 kB 00:00 2022-12-01T09:45:09.815 INFO:teuthology.orchestra.run.smithi084.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-12-01T09:45:09.890 INFO:teuthology.orchestra.run.smithi084.stdout:(118/133): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2022-12-01T09:45:09.905 INFO:teuthology.orchestra.run.smithi191.stdout:(60/133): python3-routes-2.4.1-12.el8.noarch.rp 523 kB/s | 196 kB 00:00 2022-12-01T09:45:09.915 INFO:teuthology.orchestra.run.smithi084.stdout:(119/133): python3-pytz-2017.2-9.el8.noarch.rpm 542 kB/s | 54 kB 00:00 2022-12-01T09:45:09.957 INFO:teuthology.orchestra.run.smithi084.stdout:(120/133): python3-jsonpointer-1.10-11.el8.noar 481 kB/s | 20 kB 00:00 2022-12-01T09:45:09.963 INFO:teuthology.orchestra.run.smithi191.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 130 kB/s | 24 kB 00:00 2022-12-01T09:45:09.980 INFO:teuthology.orchestra.run.smithi191.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2022-12-01T09:45:09.983 INFO:teuthology.orchestra.run.smithi084.stdout:(121/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.3 MB/s | 126 kB 00:00 2022-12-01T09:45:10.005 INFO:teuthology.orchestra.run.smithi191.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 659 kB/s | 27 kB 00:00 2022-12-01T09:45:10.041 INFO:teuthology.orchestra.run.smithi084.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2022-12-01T09:45:10.047 INFO:teuthology.orchestra.run.smithi191.stdout:(64/133): python3-kubernetes-11.0.0-6.el8.noarc 1.5 MB/s | 1.2 MB 00:00 2022-12-01T09:45:10.067 INFO:teuthology.orchestra.run.smithi084.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2022-12-01T09:45:10.089 INFO:teuthology.orchestra.run.smithi191.stdout:(65/133): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2022-12-01T09:45:10.100 INFO:teuthology.orchestra.run.smithi084.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2022-12-01T09:45:10.125 INFO:teuthology.orchestra.run.smithi084.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 462 kB/s | 27 kB 00:00 2022-12-01T09:45:10.140 INFO:teuthology.orchestra.run.smithi191.stdout:(66/133): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2022-12-01T09:45:10.165 INFO:teuthology.orchestra.run.smithi191.stdout:(67/133): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2022-12-01T09:45:10.190 INFO:teuthology.orchestra.run.smithi191.stdout:(68/133): python3-waitress-1.2.1-2.el8.1.noarch 1.1 MB/s | 241 kB 00:00 2022-12-01T09:45:10.216 INFO:teuthology.orchestra.run.smithi191.stdout:(69/133): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2022-12-01T09:45:10.241 INFO:teuthology.orchestra.run.smithi191.stdout:(70/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2022-12-01T09:45:10.268 INFO:teuthology.orchestra.run.smithi084.stdout:(126/133): python3-scipy-1.0.0-21.module+el8.5. 25 MB/s | 14 MB 00:00 2022-12-01T09:45:10.293 INFO:teuthology.orchestra.run.smithi084.stdout:(127/133): uuid-1.6.2-43.el8.x86_64.rpm 333 kB/s | 64 kB 00:00 2022-12-01T09:45:10.316 INFO:teuthology.orchestra.run.smithi191.stdout:(71/133): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2022-12-01T09:45:10.318 INFO:teuthology.orchestra.run.smithi084.stdout:(128/133): python3-pyasn1-modules-0.3.7-6.el8.n 573 kB/s | 110 kB 00:00 2022-12-01T09:45:10.342 INFO:teuthology.orchestra.run.smithi191.stdout:(72/133): fuse-2.9.7-15.el8.x86_64.rpm 826 kB/s | 83 kB 00:00 2022-12-01T09:45:10.384 INFO:teuthology.orchestra.run.smithi191.stdout:(73/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2022-12-01T09:45:10.393 INFO:teuthology.orchestra.run.smithi084.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 9.3 MB/s | 709 kB 00:00 2022-12-01T09:45:10.409 INFO:teuthology.orchestra.run.smithi191.stdout:(74/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.2 MB/s | 220 kB 00:00 2022-12-01T09:45:10.442 INFO:teuthology.orchestra.run.smithi191.stdout:(75/133): libselinux-devel-2.9-5.el8.x86_64.rpm 3.3 MB/s | 200 kB 00:00 2022-12-01T09:45:10.443 INFO:teuthology.orchestra.run.smithi084.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 29 MB/s | 1.5 MB 00:00 2022-12-01T09:45:10.467 INFO:teuthology.orchestra.run.smithi191.stdout:(76/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 311 kB/s | 18 kB 00:00 2022-12-01T09:45:10.486 INFO:teuthology.orchestra.run.smithi084.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2022-12-01T09:45:10.509 INFO:teuthology.orchestra.run.smithi191.stdout:(77/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 700 kB/s | 47 kB 00:00 2022-12-01T09:45:10.543 INFO:teuthology.orchestra.run.smithi191.stdout:(78/133): python3-cryptography-3.2.1-5.el8.x86_ 7.3 MB/s | 559 kB 00:00 2022-12-01T09:45:10.577 INFO:teuthology.orchestra.run.smithi191.stdout:(79/133): python3-oauthlib-2.1.0-1.el8.noarch.r 2.3 MB/s | 155 kB 00:00 2022-12-01T09:45:10.602 INFO:teuthology.orchestra.run.smithi191.stdout:(80/133): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2022-12-01T09:45:10.644 INFO:teuthology.orchestra.run.smithi191.stdout:(81/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 4.0 MB/s | 171 kB 00:00 2022-12-01T09:45:10.703 INFO:teuthology.orchestra.run.smithi191.stdout:(82/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 4.3 MB/s | 560 kB 00:00 2022-12-01T09:45:10.737 INFO:teuthology.orchestra.run.smithi191.stdout:(83/133): python3-jwt-1.6.1-2.el8.noarch.rpm 463 kB/s | 43 kB 00:00 2022-12-01T09:45:10.762 INFO:teuthology.orchestra.run.smithi191.stdout:(84/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2022-12-01T09:45:10.804 INFO:teuthology.orchestra.run.smithi191.stdout:(85/133): elfutils-libelf-devel-0.186-1.el8.x86 896 kB/s | 60 kB 00:00 2022-12-01T09:45:10.829 INFO:teuthology.orchestra.run.smithi191.stdout:(86/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2022-12-01T09:45:10.877 INFO:teuthology.orchestra.run.smithi084.stdout:(132/133): librbd1-18.0.0-676.gf6922ca2.el8.x86 6.2 MB/s | 3.6 MB 00:00 2022-12-01T09:45:10.879 INFO:teuthology.orchestra.run.smithi191.stdout:(87/133): thrift-0.13.0-2.el8.x86_64.rpm 2.4 MB/s | 1.7 MB 00:00 2022-12-01T09:45:10.913 INFO:teuthology.orchestra.run.smithi191.stdout:(88/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 172 kB/s | 19 kB 00:00 2022-12-01T09:45:10.938 INFO:teuthology.orchestra.run.smithi191.stdout:(89/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 911 kB/s | 98 kB 00:00 2022-12-01T09:45:10.972 INFO:teuthology.orchestra.run.smithi191.stdout:(90/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2022-12-01T09:45:10.997 INFO:teuthology.orchestra.run.smithi191.stdout:(91/133): python3-requests-oauthlib-1.0.0-1.el8 510 kB/s | 43 kB 00:00 2022-12-01T09:45:11.054 INFO:teuthology.orchestra.run.smithi191.stdout:(92/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 630 kB/s | 72 kB 00:00 2022-12-01T09:45:11.098 INFO:teuthology.orchestra.run.smithi191.stdout:(93/133): libcom_err-devel-1.45.6-4.el8.x86_64. 307 kB/s | 39 kB 00:00 2022-12-01T09:45:11.131 INFO:teuthology.orchestra.run.smithi191.stdout:(94/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 638 kB/s | 48 kB 00:00 2022-12-01T09:45:11.173 INFO:teuthology.orchestra.run.smithi191.stdout:(95/133): NetworkManager-initscripts-updown-1.3 3.2 MB/s | 138 kB 00:00 2022-12-01T09:45:11.232 INFO:teuthology.orchestra.run.smithi191.stdout:(96/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2022-12-01T09:45:11.266 INFO:teuthology.orchestra.run.smithi191.stdout:(97/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 3.8 MB/s | 644 kB 00:00 2022-12-01T09:45:11.316 INFO:teuthology.orchestra.run.smithi191.stdout:(98/133): postgresql-contrib-10.21-2.module+el8 5.6 MB/s | 808 kB 00:00 2022-12-01T09:45:11.350 INFO:teuthology.orchestra.run.smithi191.stdout:(99/133): bison-3.0.4-10.el8.x86_64.rpm 5.7 MB/s | 688 kB 00:00 2022-12-01T09:45:11.394 INFO:teuthology.orchestra.run.smithi084.stdout:(133/133): librados2-18.0.0-676.gf6922ca2.el8.x 3.2 MB/s | 3.6 MB 00:01 2022-12-01T09:45:11.397 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2022-12-01T09:45:11.397 INFO:teuthology.orchestra.run.smithi084.stdout:Total 14 MB/s | 219 MB 00:16 2022-12-01T09:45:11.424 INFO:teuthology.orchestra.run.smithi191.stdout:(100/133): postgresql-10.21-2.module+el8.6.0+15 9.7 MB/s | 1.5 MB 00:00 2022-12-01T09:45:11.483 INFO:teuthology.orchestra.run.smithi191.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2022-12-01T09:45:11.509 INFO:teuthology.orchestra.run.smithi191.stdout:(102/133): python3-markupsafe-0.23-19.el8.x86_6 1.5 MB/s | 39 kB 00:00 2022-12-01T09:45:11.575 INFO:teuthology.orchestra.run.smithi191.stdout:(103/133): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2022-12-01T09:45:11.609 INFO:teuthology.orchestra.run.smithi191.stdout:(104/133): python3-prettytable-0.7.2-14.el8.noa 439 kB/s | 44 kB 00:00 2022-12-01T09:45:11.644 INFO:teuthology.orchestra.run.smithi191.stdout:(105/133): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2022-12-01T09:45:11.718 INFO:teuthology.orchestra.run.smithi191.stdout:(106/133): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2022-12-01T09:45:11.793 INFO:teuthology.orchestra.run.smithi191.stdout:(107/133): python3-jinja2-2.10.1-3.el8.noarch.r 2.9 MB/s | 538 kB 00:00 2022-12-01T09:45:11.827 INFO:teuthology.orchestra.run.smithi191.stdout:(108/133): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2022-12-01T09:45:11.869 INFO:teuthology.orchestra.run.smithi191.stdout:(109/133): libtool-2.4.6-25.el8.x86_64.rpm 3.1 MB/s | 709 kB 00:00 2022-12-01T09:45:11.903 INFO:teuthology.orchestra.run.smithi191.stdout:(110/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.1 MB/s | 157 kB 00:00 2022-12-01T09:45:11.945 INFO:teuthology.orchestra.run.smithi191.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2022-12-01T09:45:12.028 INFO:teuthology.orchestra.run.smithi191.stdout:(112/133): openblas-threads-0.3.15-3.el8.x86_64 21 MB/s | 4.9 MB 00:00 2022-12-01T09:45:12.054 INFO:teuthology.orchestra.run.smithi191.stdout:(113/133): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2022-12-01T09:45:12.154 INFO:teuthology.orchestra.run.smithi191.stdout:(114/133): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2022-12-01T09:45:12.187 INFO:teuthology.orchestra.run.smithi191.stdout:(115/133): flex-2.6.1-9.el8.x86_64.rpm 2.0 MB/s | 320 kB 00:00 2022-12-01T09:45:12.229 INFO:teuthology.orchestra.run.smithi191.stdout:(116/133): jq-1.6-3.el8.x86_64.rpm 2.7 MB/s | 202 kB 00:00 2022-12-01T09:45:12.250 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2022-12-01T09:45:12.280 INFO:teuthology.orchestra.run.smithi191.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-12-01T09:45:12.338 INFO:teuthology.orchestra.run.smithi191.stdout:(118/133): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2022-12-01T09:45:12.380 INFO:teuthology.orchestra.run.smithi191.stdout:(119/133): python3-pytz-2017.2-9.el8.noarch.rpm 541 kB/s | 54 kB 00:00 2022-12-01T09:45:12.406 INFO:teuthology.orchestra.run.smithi191.stdout:(120/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.8 MB/s | 126 kB 00:00 2022-12-01T09:45:12.407 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2022-12-01T09:45:12.408 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2022-12-01T09:45:12.448 INFO:teuthology.orchestra.run.smithi191.stdout:(121/133): python3-jsonpointer-1.10-11.el8.noar 299 kB/s | 20 kB 00:00 2022-12-01T09:45:12.473 INFO:teuthology.orchestra.run.smithi191.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2022-12-01T09:45:12.541 INFO:teuthology.orchestra.run.smithi191.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2022-12-01T09:45:12.565 INFO:teuthology.orchestra.run.smithi191.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2022-12-01T09:45:12.599 INFO:teuthology.orchestra.run.smithi191.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 452 kB/s | 27 kB 00:00 2022-12-01T09:45:12.624 INFO:teuthology.orchestra.run.smithi191.stdout:(126/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-12-01T09:45:12.758 INFO:teuthology.orchestra.run.smithi191.stdout:(127/133): python3-scipy-1.0.0-21.module+el8.5. 24 MB/s | 14 MB 00:00 2022-12-01T09:45:12.791 INFO:teuthology.orchestra.run.smithi191.stdout:(128/133): python3-pyasn1-modules-0.3.7-6.el8.n 573 kB/s | 110 kB 00:00 2022-12-01T09:45:12.834 INFO:teuthology.orchestra.run.smithi191.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 17 MB/s | 709 kB 00:00 2022-12-01T09:45:12.917 INFO:teuthology.orchestra.run.smithi191.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 18 MB/s | 1.5 MB 00:00 2022-12-01T09:45:12.959 INFO:teuthology.orchestra.run.smithi191.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2022-12-01T09:45:13.235 INFO:teuthology.orchestra.run.smithi191.stdout:(132/133): librados2-18.0.0-676.gf6922ca2.el8.x 5.9 MB/s | 3.6 MB 00:00 2022-12-01T09:45:13.551 INFO:teuthology.orchestra.run.smithi191.stdout:(133/133): librbd1-18.0.0-676.gf6922ca2.el8.x86 4.5 MB/s | 3.6 MB 00:00 2022-12-01T09:45:13.554 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2022-12-01T09:45:13.554 INFO:teuthology.orchestra.run.smithi191.stdout:Total 13 MB/s | 219 MB 00:16 2022-12-01T09:45:13.583 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2022-12-01T09:45:13.587 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2022-12-01T09:45:14.398 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2022-12-01T09:45:14.550 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2022-12-01T09:45:14.551 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2022-12-01T09:45:15.711 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2022-12-01T09:45:15.715 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2022-12-01T09:45:18.711 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2022-12-01T09:45:18.907 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-12-01T09:45:19.156 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-12-01T09:45:19.342 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-12-01T09:45:19.465 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-12-01T09:45:19.617 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-12-01T09:45:19.778 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-12-01T09:45:19.875 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-12-01T09:45:20.045 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-12-01T09:45:20.441 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-12-01T09:45:20.686 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:45:20.839 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2022-12-01T09:45:21.072 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:45:21.145 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-12-01T09:45:21.236 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:45:21.360 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:45:21.364 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-12-01T09:45:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-12-01T09:45:21.553 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:45:21.648 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:45:21.690 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-12-01T09:45:21.833 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 10/138 2022-12-01T09:45:21.866 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-12-01T09:45:22.044 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-12-01T09:45:22.105 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-12-01T09:45:22.132 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-12-01T09:45:22.247 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-12-01T09:45:22.285 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-12-01T09:45:22.611 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-12-01T09:45:22.697 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-12-01T09:45:22.823 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-12-01T09:45:22.927 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:45:22.969 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:45:23.071 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:45:23.261 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 16/138 2022-12-01T09:45:23.319 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:45:23.335 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 16/138 2022-12-01T09:45:23.527 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:45:23.652 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:45:23.815 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-12-01T09:45:23.893 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:45:23.999 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:45:24.006 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-12-01T09:45:24.175 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-12-01T09:45:24.214 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 10/138 2022-12-01T09:45:24.314 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-12-01T09:45:24.395 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-12-01T09:45:24.437 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-12-01T09:45:24.563 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-12-01T09:45:24.630 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-12-01T09:45:24.677 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-12-01T09:45:24.969 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-12-01T09:45:25.197 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-12-01T09:45:25.279 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-12-01T09:45:25.369 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:45:25.435 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-12-01T09:45:25.471 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:45:25.653 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 16/138 2022-12-01T09:45:25.718 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 16/138 2022-12-01T09:45:25.871 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-12-01T09:45:26.060 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-12-01T09:45:26.100 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-12-01T09:45:26.251 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-12-01T09:45:26.565 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-12-01T09:45:26.715 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-12-01T09:45:26.859 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-12-01T09:45:26.916 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-12-01T09:45:26.992 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-12-01T09:45:27.143 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-12-01T09:45:27.206 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-12-01T09:45:27.442 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-12-01T09:45:27.608 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x 27/138 2022-12-01T09:45:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8 28/138 2022-12-01T09:45:27.762 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-12-01T09:45:27.967 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-12-01T09:45:27.988 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 29/138 2022-12-01T09:45:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-12-01T09:45:28.560 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-12-01T09:45:28.743 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-12-01T09:45:29.415 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-12-01T09:45:29.739 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-12-01T09:45:29.871 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-12-01T09:45:29.980 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-12-01T09:45:30.157 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x 27/138 2022-12-01T09:45:30.244 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-12-01T09:45:30.294 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8 28/138 2022-12-01T09:45:30.571 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 29/138 2022-12-01T09:45:30.784 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-12-01T09:45:31.214 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-12-01T09:45:31.536 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-12-01T09:45:31.801 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-12-01T09:45:31.994 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-12-01T09:45:32.201 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-12-01T09:45:32.456 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 37/138 2022-12-01T09:45:32.500 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-12-01T09:45:32.705 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 38/138 2022-12-01T09:45:32.867 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-12-01T09:45:32.902 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-12-01T09:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-12-01T09:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-12-01T09:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-12-01T09:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-12-01T09:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-12-01T09:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-12-01T09:45:33.877 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-12-01T09:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-12-01T09:45:34.219 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-12-01T09:45:34.250 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-12-01T09:45:34.388 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-12-01T09:45:34.508 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-12-01T09:45:34.529 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-12-01T09:45:34.678 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-12-01T09:45:34.736 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-12-01T09:45:34.891 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-12-01T09:45:34.960 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-12-01T09:45:35.072 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-12-01T09:45:35.206 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 37/138 2022-12-01T09:45:35.320 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-12-01T09:45:35.397 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 38/138 2022-12-01T09:45:35.602 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-12-01T09:45:35.734 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-12-01T09:45:35.775 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-12-01T09:45:35.847 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-12-01T09:45:36.041 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-12-01T09:45:36.060 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-12-01T09:45:36.121 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-12-01T09:45:36.201 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-12-01T09:45:36.332 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-12-01T09:45:36.451 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-12-01T09:45:36.500 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-12-01T09:45:36.711 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-12-01T09:45:36.757 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-12-01T09:45:36.949 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-12-01T09:45:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-12-01T09:45:37.137 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-12-01T09:45:37.279 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-12-01T09:45:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-12-01T09:45:37.467 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-12-01T09:45:37.470 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-12-01T09:45:37.673 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-12-01T09:45:37.703 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-12-01T09:45:37.827 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-12-01T09:45:37.897 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-12-01T09:45:38.113 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-12-01T09:45:38.140 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-12-01T09:45:38.340 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-12-01T09:45:38.505 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-12-01T09:45:38.578 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-12-01T09:45:38.808 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-12-01T09:45:38.839 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-12-01T09:45:39.001 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-12-01T09:45:39.117 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-12-01T09:45:39.260 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-12-01T09:45:39.295 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-12-01T09:45:39.475 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-12-01T09:45:39.477 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-12-01T09:45:39.653 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-12-01T09:45:39.829 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-12-01T09:45:39.956 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-12-01T09:45:40.104 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-12-01T09:45:40.180 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-12-01T09:45:40.250 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-12-01T09:45:40.337 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-12-01T09:45:40.509 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-12-01T09:45:40.518 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-12-01T09:45:40.690 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-12-01T09:45:40.701 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-12-01T09:45:40.910 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-12-01T09:45:41.024 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-12-01T09:45:41.067 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-12-01T09:45:41.211 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-12-01T09:45:41.251 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-12-01T09:45:41.379 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-12-01T09:45:41.477 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-12-01T09:45:41.617 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-12-01T09:45:41.719 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-12-01T09:45:41.809 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-12-01T09:45:41.966 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-12-01T09:45:41.987 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-12-01T09:45:42.182 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-12-01T09:45:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-12-01T09:45:42.323 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-12-01T09:45:42.397 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-12-01T09:45:42.543 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-12-01T09:45:42.588 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-12-01T09:45:42.771 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-12-01T09:45:42.812 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-12-01T09:45:43.022 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-12-01T09:45:43.219 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-12-01T09:45:43.384 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-12-01T09:45:43.607 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-12-01T09:45:43.819 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-12-01T09:45:43.842 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-12-01T09:45:44.034 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-12-01T09:45:44.100 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-12-01T09:45:44.184 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-12-01T09:45:44.337 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-12-01T09:45:44.352 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-12-01T09:45:44.544 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-12-01T09:45:44.662 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-12-01T09:45:44.775 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-12-01T09:45:44.897 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-12-01T09:45:44.937 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-12-01T09:45:45.080 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-12-01T09:45:45.208 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-12-01T09:45:45.438 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-12-01T09:45:45.629 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-12-01T09:45:45.826 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-12-01T09:45:45.869 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-12-01T09:45:46.005 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 93/138 2022-12-01T09:45:46.080 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 93/138 2022-12-01T09:45:46.239 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 94/138 2022-12-01T09:45:46.886 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-12-01T09:45:47.534 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-12-01T09:45:47.799 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-12-01T09:45:48.079 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-12-01T09:45:48.373 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-12-01T09:45:48.580 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-12-01T09:45:49.335 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-12-01T09:45:49.473 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 93/138 2022-12-01T09:45:49.562 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 93/138 2022-12-01T09:45:49.748 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 94/138 2022-12-01T09:45:50.077 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:45:50.255 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:45:50.809 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:45:51.042 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 96/138 2022-12-01T09:45:51.140 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 96/138 2022-12-01T09:45:51.237 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 97/138 2022-12-01T09:45:53.677 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:45:53.872 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:45:54.425 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 95/138 2022-12-01T09:45:54.748 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 96/138 2022-12-01T09:45:54.826 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 96/138 2022-12-01T09:45:54.946 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 97/138 2022-12-01T09:46:10.101 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 97/138 2022-12-01T09:46:10.102 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2022-12-01T09:46:10.102 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2022-12-01T09:46:10.103 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /dev 2022-12-01T09:46:10.103 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /run 2022-12-01T09:46:10.103 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2022-12-01T09:46:10.103 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2022-12-01T09:46:10.104 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2022-12-01T09:46:10.104 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2022-12-01T09:46:10.104 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /dev 2022-12-01T09:46:10.104 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:10.492 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 98/138 2022-12-01T09:46:11.973 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 98/138 2022-12-01T09:46:12.157 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 99/138 2022-12-01T09:46:12.375 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 99/138 2022-12-01T09:46:12.533 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 100/138 2022-12-01T09:46:12.917 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 100/138 2022-12-01T09:46:13.058 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 101/138 2022-12-01T09:46:13.075 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 101/138 2022-12-01T09:46:13.818 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 97/138 2022-12-01T09:46:13.818 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2022-12-01T09:46:13.818 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2022-12-01T09:46:13.819 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2022-12-01T09:46:13.819 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /run 2022-12-01T09:46:13.819 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2022-12-01T09:46:13.819 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2022-12-01T09:46:13.820 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2022-12-01T09:46:13.820 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2022-12-01T09:46:13.820 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2022-12-01T09:46:13.820 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:14.226 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 98/138 2022-12-01T09:46:14.716 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 102/138 2022-12-01T09:46:14.839 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 102/138 2022-12-01T09:46:14.980 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el 103/138 2022-12-01T09:46:15.144 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.e 104/138 2022-12-01T09:46:15.332 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 105/138 2022-12-01T09:46:15.538 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 105/138 2022-12-01T09:46:15.710 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 98/138 2022-12-01T09:46:15.844 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 106/138 2022-12-01T09:46:15.911 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 99/138 2022-12-01T09:46:16.127 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 99/138 2022-12-01T09:46:16.276 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 100/138 2022-12-01T09:46:16.657 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 100/138 2022-12-01T09:46:16.801 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 106/138 2022-12-01T09:46:16.826 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 101/138 2022-12-01T09:46:16.842 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 101/138 2022-12-01T09:46:16.932 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 107/138 2022-12-01T09:46:17.020 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 107/138 2022-12-01T09:46:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8 108/138 2022-12-01T09:46:17.505 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 109/138 2022-12-01T09:46:17.540 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 109/138 2022-12-01T09:46:17.723 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 110/138 2022-12-01T09:46:17.735 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 110/138 2022-12-01T09:46:18.605 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 102/138 2022-12-01T09:46:18.755 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 102/138 2022-12-01T09:46:18.915 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el 103/138 2022-12-01T09:46:19.078 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.e 104/138 2022-12-01T09:46:19.267 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 105/138 2022-12-01T09:46:19.413 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 111/138 2022-12-01T09:46:19.475 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 105/138 2022-12-01T09:46:19.592 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 112/138 2022-12-01T09:46:19.626 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 112/138 2022-12-01T09:46:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 106/138 2022-12-01T09:46:19.773 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-676.gf6922c 113/138 2022-12-01T09:46:19.815 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-676.gf6922c 113/138 2022-12-01T09:46:19.940 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 114/138 2022-12-01T09:46:20.286 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 114/138 2022-12-01T09:46:20.448 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 115/138 2022-12-01T09:46:20.726 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 106/138 2022-12-01T09:46:20.875 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 107/138 2022-12-01T09:46:20.964 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 107/138 2022-12-01T09:46:21.360 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8 108/138 2022-12-01T09:46:21.577 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 109/138 2022-12-01T09:46:21.612 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 109/138 2022-12-01T09:46:21.783 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 110/138 2022-12-01T09:46:21.794 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 110/138 2022-12-01T09:46:23.474 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 111/138 2022-12-01T09:46:23.683 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 112/138 2022-12-01T09:46:23.717 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 112/138 2022-12-01T09:46:23.841 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-676.gf6922c 113/138 2022-12-01T09:46:23.883 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-676.gf6922c 113/138 2022-12-01T09:46:24.041 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 114/138 2022-12-01T09:46:24.388 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 114/138 2022-12-01T09:46:24.524 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 115/138 2022-12-01T09:46:25.532 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 115/138 2022-12-01T09:46:25.802 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 116/138 2022-12-01T09:46:26.573 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 117/138 2022-12-01T09:46:26.824 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-12-01T09:46:27.023 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-12-01T09:46:27.188 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-12-01T09:46:27.376 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-12-01T09:46:27.617 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-12-01T09:46:27.800 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_6 122/138 2022-12-01T09:46:27.947 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-12-01T09:46:29.615 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 115/138 2022-12-01T09:46:29.929 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 116/138 2022-12-01T09:46:30.009 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:46:30.301 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:46:30.319 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:46:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 125/138 2022-12-01T09:46:30.705 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 117/138 2022-12-01T09:46:30.728 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 126/138 2022-12-01T09:46:30.983 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-12-01T09:46:30.993 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 127/138 2022-12-01T09:46:31.208 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-12-01T09:46:31.316 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-12-01T09:46:31.390 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-12-01T09:46:31.578 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-12-01T09:46:31.648 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-12-01T09:46:31.836 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-12-01T09:46:31.870 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-12-01T09:46:31.945 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-12-01T09:46:32.052 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_6 122/138 2022-12-01T09:46:32.128 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-12-01T09:46:32.140 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-12-01T09:46:32.207 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-12-01T09:46:32.312 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-12-01T09:46:32.320 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-12-01T09:46:32.432 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-12-01T09:46:32.639 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-12-01T09:46:32.639 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-12-01T09:46:32.798 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-12-01T09:46:32.799 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-12-01T09:46:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-12-01T09:46:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-12-01T09:46:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-12-01T09:46:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:46:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:46:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 1/138 2022-12-01T09:46:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 2/138 2022-12-01T09:46:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 3/138 2022-12-01T09:46:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 4/138 2022-12-01T09:46:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-676.gf6922c 5/138 2022-12-01T09:46:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 6/138 2022-12-01T09:46:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:46:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:46:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:46:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 10/138 2022-12-01T09:46:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 11/138 2022-12-01T09:46:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 12/138 2022-12-01T09:46:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 13/138 2022-12-01T09:46:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_6 14/138 2022-12-01T09:46:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:46:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 16/138 2022-12-01T09:46:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 17/138 2022-12-01T09:46:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 18/138 2022-12-01T09:46:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 19/138 2022-12-01T09:46:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8 20/138 2022-12-01T09:46:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x 21/138 2022-12-01T09:46:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 22/138 2022-12-01T09:46:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 23/138 2022-12-01T09:46:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 24/138 2022-12-01T09:46:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 25/138 2022-12-01T09:46:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 26/138 2022-12-01T09:46:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 27/138 2022-12-01T09:46:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 28/138 2022-12-01T09:46:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.e 29/138 2022-12-01T09:46:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 30/138 2022-12-01T09:46:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 31/138 2022-12-01T09:46:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 32/138 2022-12-01T09:46:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8 33/138 2022-12-01T09:46:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 34/138 2022-12-01T09:46:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el 35/138 2022-12-01T09:46:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 36/138 2022-12-01T09:46:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 37/138 2022-12-01T09:46:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 38/138 2022-12-01T09:46:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-12-01T09:46:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-12-01T09:46:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-12-01T09:46:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-12-01T09:46:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-12-01T09:46:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-12-01T09:46:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-12-01T09:46:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-12-01T09:46:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-12-01T09:46:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-12-01T09:46:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-12-01T09:46:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-12-01T09:46:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-12-01T09:46:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-12-01T09:46:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-12-01T09:46:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-12-01T09:46:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-12-01T09:46:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-12-01T09:46:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-12-01T09:46:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-12-01T09:46:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-12-01T09:46:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-12-01T09:46:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-12-01T09:46:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-12-01T09:46:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-12-01T09:46:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-12-01T09:46:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-12-01T09:46:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-12-01T09:46:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-12-01T09:46:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-12-01T09:46:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-12-01T09:46:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-12-01T09:46:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-12-01T09:46:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-12-01T09:46:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-12-01T09:46:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-12-01T09:46:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-12-01T09:46:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-12-01T09:46:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-12-01T09:46:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-12-01T09:46:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-12-01T09:46:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-12-01T09:46:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-12-01T09:46:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-12-01T09:46:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-12-01T09:46:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-12-01T09:46:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-12-01T09:46:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-12-01T09:46:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-12-01T09:46:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-12-01T09:46:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-12-01T09:46:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 90/138 2022-12-01T09:46:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 91/138 2022-12-01T09:46:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 92/138 2022-12-01T09:46:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 93/138 2022-12-01T09:46:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 94/138 2022-12-01T09:46:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 95/138 2022-12-01T09:46:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/138 2022-12-01T09:46:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/138 2022-12-01T09:46:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-12-01T09:46:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-12-01T09:46:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-12-01T09:46:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-12-01T09:46:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-12-01T09:46:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-12-01T09:46:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-12-01T09:46:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-12-01T09:46:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-12-01T09:46:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-12-01T09:46:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-12-01T09:46:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-12-01T09:46:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-12-01T09:46:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-12-01T09:46:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-12-01T09:46:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-12-01T09:46:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-12-01T09:46:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-12-01T09:46:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-12-01T09:46:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-12-01T09:46:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-12-01T09:46:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-12-01T09:46:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-12-01T09:46:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-12-01T09:46:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-12-01T09:46:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-12-01T09:46:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-12-01T09:46:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-12-01T09:46:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-12-01T09:46:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-12-01T09:46:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-12-01T09:46:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 129/138 2022-12-01T09:46:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-12-01T09:46:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 131/138 2022-12-01T09:46:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-12-01T09:46:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-12-01T09:46:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-12-01T09:46:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-12-01T09:46:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-12-01T09:46:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-12-01T09:46:34.324 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:46:34.644 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:46:34.661 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-12-01T09:46:34.916 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 125/138 2022-12-01T09:46:35.063 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 126/138 2022-12-01T09:46:35.304 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 127/138 2022-12-01T09:46:35.670 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-12-01T09:46:35.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:46:35.967 INFO:teuthology.orchestra.run.smithi084.stdout:Installed products updated. 2022-12-01T09:46:36.049 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-12-01T09:46:36.213 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:36.213 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2022-12-01T09:46:36.214 INFO:teuthology.orchestra.run.smithi084.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-12-01T09:46:36.214 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.214 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.215 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:46:36.215 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:46:36.215 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2022-12-01T09:46:36.215 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-12-01T09:46:36.216 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.1-7.el8.noarch 2022-12-01T09:46:36.216 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.0.4-10.el8.x86_64 2022-12-01T09:46:36.216 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.216 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.217 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.217 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.217 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.217 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.218 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.218 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.218 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.218 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.219 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.219 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.219 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.219 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-12-01T09:46:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.1-9.el8.x86_64 2022-12-01T09:46:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.7-15.el8.x86_64 2022-12-01T09:46:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-12-01T09:46:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: jq-1.6-3.el8.x86_64 2022-12-01T09:46:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-12-01T09:46:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-12-01T09:46:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-12-01T09:46:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-12-01T09:46:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-12-01T09:46:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.2-3.el8.x86_64 2022-12-01T09:46:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: libpq-13.5-1.el8.x86_64 2022-12-01T09:46:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-12-01T09:46:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-12-01T09:46:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-12-01T09:46:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-12-01T09:46:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-12-01T09:46:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-25.el8.x86_64 2022-12-01T09:46:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.3.1-3.el8.x86_64 2022-12-01T09:46:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-12-01T09:46:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-12-01T09:46:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-12-01T09:46:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-12-01T09:46:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.15-3.el8.x86_64 2022-12-01T09:46:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-12-01T09:46:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:46:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-12-01T09:46:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-12-01T09:46:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-12-01T09:46:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:46:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:46:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:46:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-12-01T09:46:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.5.1-7.el8.noarch 2022-12-01T09:46:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-12-01T09:46:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-12-01T09:46:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-12-01T09:46:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-12-01T09:46:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-12-01T09:46:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-12-01T09:46:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-12-01T09:46:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-12-01T09:46:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-12-01T09:46:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-6.2-6.el8.noarch 2022-12-01T09:46:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-12-01T09:46:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-12-01T09:46:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-12-01T09:46:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-12-01T09:46:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-12-01T09:46:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-12-01T09:46:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-12-01T09:46:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.0.6-13.el8.noarch 2022-12-01T09:46:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-12-01T09:46:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-12-01T09:46:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-12-01T09:46:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-12-01T09:46:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-12-01T09:46:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-12-01T09:46:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-12-01T09:46:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-2.6-1.el8.noarch 2022-12-01T09:46:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-12-01T09:46:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-12-01T09:46:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-12-01T09:46:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-12-01T09:46:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.14-14.el8.noarch 2022-12-01T09:46:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2017.2-9.el8.noarch 2022-12-01T09:46:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-12-01T09:46:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-12-01T09:46:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.4.1-12.el8.noarch 2022-12-01T09:46:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.8-1.el8.noarch 2022-12-01T09:46:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-12-01T09:46:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-12-01T09:46:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-12-01T09:46:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-12-01T09:46:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-12-01T09:46:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-12-01T09:46:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-12-01T09:46:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-12-01T09:46:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-12-01T09:46:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-12-01T09:46:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-12-01T09:46:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-1.el8.x86_64 2022-12-01T09:46:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.13.0-2.el8.x86_64 2022-12-01T09:46:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-43.el8.x86_64 2022-12-01T09:46:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-12-01T09:46:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-12-01T09:46:36.246 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-12-01T09:46:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-12-01T09:46:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:36.247 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2022-12-01T09:46:36.305 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-12-01T09:46:36.402 DEBUG:teuthology.parallel:result is None 2022-12-01T09:46:36.539 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-12-01T09:46:36.551 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-12-01T09:46:36.739 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-12-01T09:46:36.746 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-12-01T09:46:36.883 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-12-01T09:46:37.049 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-12-01T09:46:37.049 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-12-01T09:46:37.259 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-12-01T09:46:37.259 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-12-01T09:46:37.500 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-12-01T09:46:37.500 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-12-01T09:46:37.728 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-12-01T09:46:37.937 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:46:38.462 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:46:38.462 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 1/138 2022-12-01T09:46:38.462 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 2/138 2022-12-01T09:46:38.463 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 3/138 2022-12-01T09:46:38.463 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 4/138 2022-12-01T09:46:38.463 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-676.gf6922c 5/138 2022-12-01T09:46:38.463 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 6/138 2022-12-01T09:46:38.464 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 7/138 2022-12-01T09:46:38.464 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 8/138 2022-12-01T09:46:38.464 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 9/138 2022-12-01T09:46:38.464 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 10/138 2022-12-01T09:46:38.465 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 11/138 2022-12-01T09:46:38.465 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 12/138 2022-12-01T09:46:38.465 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 13/138 2022-12-01T09:46:38.465 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_6 14/138 2022-12-01T09:46:38.466 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 15/138 2022-12-01T09:46:38.466 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 16/138 2022-12-01T09:46:38.466 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 17/138 2022-12-01T09:46:38.466 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_ 18/138 2022-12-01T09:46:38.467 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 19/138 2022-12-01T09:46:38.467 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8 20/138 2022-12-01T09:46:38.467 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x 21/138 2022-12-01T09:46:38.467 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 22/138 2022-12-01T09:46:38.468 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 23/138 2022-12-01T09:46:38.468 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 24/138 2022-12-01T09:46:38.468 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 25/138 2022-12-01T09:46:38.468 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 26/138 2022-12-01T09:46:38.469 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 27/138 2022-12-01T09:46:38.469 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 28/138 2022-12-01T09:46:38.482 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.e 29/138 2022-12-01T09:46:38.483 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noar 30/138 2022-12-01T09:46:38.483 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.no 31/138 2022-12-01T09:46:38.483 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-676.gf692 32/138 2022-12-01T09:46:38.483 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8 33/138 2022-12-01T09:46:38.484 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 34/138 2022-12-01T09:46:38.484 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el 35/138 2022-12-01T09:46:38.484 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 36/138 2022-12-01T09:46:38.484 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 37/138 2022-12-01T09:46:38.485 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 38/138 2022-12-01T09:46:38.485 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-12-01T09:46:38.485 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-12-01T09:46:38.485 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-12-01T09:46:38.486 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-12-01T09:46:38.486 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-12-01T09:46:38.486 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-12-01T09:46:38.486 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-12-01T09:46:38.486 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-12-01T09:46:38.487 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-12-01T09:46:38.487 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-12-01T09:46:38.487 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-12-01T09:46:38.487 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-12-01T09:46:38.488 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-12-01T09:46:38.488 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-12-01T09:46:38.488 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-12-01T09:46:38.488 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-12-01T09:46:38.489 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-12-01T09:46:38.489 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-12-01T09:46:38.489 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-12-01T09:46:38.489 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-12-01T09:46:38.490 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-12-01T09:46:38.490 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-12-01T09:46:38.490 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-12-01T09:46:38.490 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-12-01T09:46:38.490 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-12-01T09:46:38.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-12-01T09:46:38.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-12-01T09:46:38.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-12-01T09:46:38.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-12-01T09:46:38.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-12-01T09:46:38.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-12-01T09:46:38.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-12-01T09:46:38.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-12-01T09:46:38.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-12-01T09:46:38.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-12-01T09:46:38.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-12-01T09:46:38.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-12-01T09:46:38.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-12-01T09:46:38.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-12-01T09:46:38.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-12-01T09:46:38.494 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-12-01T09:46:38.495 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-12-01T09:46:38.495 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-12-01T09:46:38.495 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-12-01T09:46:38.495 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-12-01T09:46:38.496 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-12-01T09:46:38.496 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-12-01T09:46:38.497 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-12-01T09:46:38.497 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-12-01T09:46:38.497 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-12-01T09:46:38.498 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-12-01T09:46:38.498 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 90/138 2022-12-01T09:46:38.498 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 91/138 2022-12-01T09:46:38.498 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 92/138 2022-12-01T09:46:38.499 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 93/138 2022-12-01T09:46:38.499 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 94/138 2022-12-01T09:46:38.499 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 95/138 2022-12-01T09:46:38.499 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/138 2022-12-01T09:46:38.500 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/138 2022-12-01T09:46:38.500 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-12-01T09:46:38.500 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-12-01T09:46:38.501 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-12-01T09:46:38.501 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-12-01T09:46:38.501 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-12-01T09:46:38.502 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-12-01T09:46:38.502 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-12-01T09:46:38.502 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-12-01T09:46:38.502 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-12-01T09:46:38.503 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-12-01T09:46:38.503 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-12-01T09:46:38.503 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-12-01T09:46:38.503 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-12-01T09:46:38.504 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-12-01T09:46:38.504 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-12-01T09:46:38.504 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-12-01T09:46:38.504 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-12-01T09:46:38.505 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-12-01T09:46:38.505 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-12-01T09:46:38.505 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-12-01T09:46:38.505 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-12-01T09:46:38.506 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-12-01T09:46:38.506 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-12-01T09:46:38.506 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-12-01T09:46:38.506 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-12-01T09:46:38.506 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-12-01T09:46:38.507 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-12-01T09:46:38.507 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-12-01T09:46:38.507 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-12-01T09:46:38.507 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-12-01T09:46:38.508 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-12-01T09:46:38.508 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 129/138 2022-12-01T09:46:38.508 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-12-01T09:46:38.508 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 131/138 2022-12-01T09:46:38.509 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-12-01T09:46:38.509 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-12-01T09:46:38.509 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-12-01T09:46:38.509 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-12-01T09:46:38.510 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-12-01T09:46:38.510 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-12-01T09:46:40.453 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-12-01T09:46:40.453 INFO:teuthology.orchestra.run.smithi191.stdout:Installed products updated. 2022-12-01T09:46:40.711 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:40.711 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2022-12-01T09:46:40.711 INFO:teuthology.orchestra.run.smithi191.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-12-01T09:46:40.712 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.712 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.712 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:46:40.712 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:46:40.712 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2022-12-01T09:46:40.713 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-12-01T09:46:40.713 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.1-7.el8.noarch 2022-12-01T09:46:40.713 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.0.4-10.el8.x86_64 2022-12-01T09:46:40.713 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.714 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.714 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.714 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.714 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.715 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.715 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.715 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.715 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.716 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.716 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.716 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.716 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.717 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.717 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.717 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.717 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.717 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.718 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.718 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.718 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.718 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.718 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:18.0.0-676.gf6922ca2.el8.noarch 2022-12-01T09:46:40.719 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-12-01T09:46:40.719 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.1-9.el8.x86_64 2022-12-01T09:46:40.719 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.7-15.el8.x86_64 2022-12-01T09:46:40.719 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-12-01T09:46:40.720 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.6-3.el8.x86_64 2022-12-01T09:46:40.720 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-12-01T09:46:40.720 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-12-01T09:46:40.720 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-12-01T09:46:40.721 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.721 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.721 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.721 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-12-01T09:46:40.721 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-12-01T09:46:40.722 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.2-3.el8.x86_64 2022-12-01T09:46:40.722 INFO:teuthology.orchestra.run.smithi191.stdout: libpq-13.5-1.el8.x86_64 2022-12-01T09:46:40.722 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-12-01T09:46:40.722 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-12-01T09:46:40.723 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.723 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.723 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-12-01T09:46:40.723 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.724 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-12-01T09:46:40.724 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-12-01T09:46:40.724 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-25.el8.x86_64 2022-12-01T09:46:40.724 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.3.1-3.el8.x86_64 2022-12-01T09:46:40.724 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-12-01T09:46:40.725 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-12-01T09:46:40.725 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-12-01T09:46:40.725 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-12-01T09:46:40.725 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.15-3.el8.x86_64 2022-12-01T09:46:40.725 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-12-01T09:46:40.726 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-12-01T09:46:40.726 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-12-01T09:46:40.726 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-12-01T09:46:40.726 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-12-01T09:46:40.727 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:46:40.727 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:46:40.727 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-12-01T09:46:40.727 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-12-01T09:46:40.728 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.5.1-7.el8.noarch 2022-12-01T09:46:40.728 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-12-01T09:46:40.728 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-12-01T09:46:40.728 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-12-01T09:46:40.729 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.729 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.729 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.729 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-12-01T09:46:40.729 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-12-01T09:46:40.730 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-12-01T09:46:40.730 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-12-01T09:46:40.730 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-12-01T09:46:40.730 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-12-01T09:46:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-6.2-6.el8.noarch 2022-12-01T09:46:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-12-01T09:46:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-12-01T09:46:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-12-01T09:46:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-12-01T09:46:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-12-01T09:46:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-12-01T09:46:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-12-01T09:46:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.0.6-13.el8.noarch 2022-12-01T09:46:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-12-01T09:46:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-12-01T09:46:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-12-01T09:46:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-12-01T09:46:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-12-01T09:46:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-12-01T09:46:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-12-01T09:46:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-2.6-1.el8.noarch 2022-12-01T09:46:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-12-01T09:46:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-12-01T09:46:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-12-01T09:46:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-12-01T09:46:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.14-14.el8.noarch 2022-12-01T09:46:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2017.2-9.el8.noarch 2022-12-01T09:46:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-12-01T09:46:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-12-01T09:46:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.4.1-12.el8.noarch 2022-12-01T09:46:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.8-1.el8.noarch 2022-12-01T09:46:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-12-01T09:46:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-12-01T09:46:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-12-01T09:46:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-12-01T09:46:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-12-01T09:46:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-12-01T09:46:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-12-01T09:46:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-12-01T09:46:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-12-01T09:46:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-12-01T09:46:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-12-01T09:46:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:18.0.0-676.gf6922ca2.el8.x86_64 2022-12-01T09:46:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-1.el8.x86_64 2022-12-01T09:46:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.13.0-2.el8.x86_64 2022-12-01T09:46:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-43.el8.x86_64 2022-12-01T09:46:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-12-01T09:46:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-12-01T09:46:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-12-01T09:46:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:40.742 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2022-12-01T09:46:40.919 DEBUG:teuthology.parallel:result is None 2022-12-01T09:46:40.920 DEBUG:teuthology.orchestra.run.smithi084:> sudo lsb_release -is 2022-12-01T09:46:40.986 INFO:teuthology.orchestra.run.smithi084.stdout:RedHatEnterprise 2022-12-01T09:46:40.988 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:46:40.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:46:41.144 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-12-01T09:46:41.169 INFO:teuthology.orchestra.run.smithi084.stdout:18.0.0-676.gf6922ca2.el8 2022-12-01T09:46:41.170 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-676.gf6922ca2.el8 2022-12-01T09:46:41.171 INFO:teuthology.task.install:The correct ceph version 18.0.0-676.gf6922ca2 is installed. 2022-12-01T09:46:41.172 DEBUG:teuthology.orchestra.run.smithi191:> sudo lsb_release -is 2022-12-01T09:46:41.227 INFO:teuthology.orchestra.run.smithi191.stdout:RedHatEnterprise 2022-12-01T09:46:41.229 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-12-01T09:46:41.230 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f6922ca21f63dad2b198f92826f75d268f12fc97 2022-12-01T09:46:41.385 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-12-01T09:46:41.407 INFO:teuthology.orchestra.run.smithi191.stdout:18.0.0-676.gf6922ca2.el8 2022-12-01T09:46:41.408 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-676.gf6922ca2.el8 2022-12-01T09:46:41.408 INFO:teuthology.task.install:The correct ceph version 18.0.0-676.gf6922ca2 is installed. 2022-12-01T09:46:41.409 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-12-01T09:46:41.409 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:41.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-12-01T09:46:41.441 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:46:41.441 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-12-01T09:46:41.475 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-12-01T09:46:41.476 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:41.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2022-12-01T09:46:41.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-12-01T09:46:41.578 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:46:41.579 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2022-12-01T09:46:41.606 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-12-01T09:46:41.673 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-12-01T09:46:41.674 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:41.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2022-12-01T09:46:41.703 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-12-01T09:46:41.772 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:46:41.772 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2022-12-01T09:46:41.799 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-12-01T09:46:41.865 INFO:teuthology.run_tasks:Running task ceph... 2022-12-01T09:46:41.935 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-12-01T09:46:41.936 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /var/log/ceph 2022-12-01T09:46:41.939 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2022-12-01T09:46:41.965 INFO:tasks.ceph:Disabling ceph logrotate... 2022-12-01T09:46:41.965 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/logrotate.d/ceph 2022-12-01T09:46:42.007 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2022-12-01T09:46:42.034 INFO:tasks.ceph:Creating extra log directories... 2022-12-01T09:46:42.035 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-12-01T09:46:42.074 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-12-01T09:46:42.107 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-12-01T09:46:42.107 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-12-01T09:46:42.108 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-12-01T09:46:42.108 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:42.108 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-12-01T09:46:42.155 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-12-01T09:46:42.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-12-01T09:46:42.293 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:46:42.294 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-12-01T09:46:42.312 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-12-01T09:46:42.431 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-12-01T09:46:42.464 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-12-01T09:46:42.465 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-12-01T09:46:42.465 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi/7090979', 'branch': 'wip-khiremat-47893-test-mgr-finisher-thread-5', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7090979', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'khiremat-2022-11-24_05:02:42-fs:volumes-wip-khiremat-47893-test-mgr-finisher-thread-5-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-khiremat-47893-test-mgr-finisher-thread-5'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-khiremat-47893-test-mgr-finisher-thread-5', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}}, 'owner': 'scheduled_khiremat@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'wip-khiremat-47893-test-mgr-finisher-thread-5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_f6922ca21f63dad2b198f92826f75d268f12fc97/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi084.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaq8FNWQ+dGpbsVxrhRBQus5ucOH55AdmRfW6SAsawZe5S7f5ymoeSxB4tLgMGB5elAIJb3WSTwVg5JQU+vSXsqzVUNzvMpu0Ifz2xMA2xsN3k6Ql3bQhtsKGDU+96vHMsk2JZXQBlROzbtbpy9Jpm6vMzBGyWZc91VUzaHZkGlaMPdEQ7NrgcGkX9DdI1x3XmUM2D1TUUrQXPQrY4y5W2MOt7dGQi7hEjkuabvVxHcqq+dnZ/Nyol3JUJ6lupNzrjPv1QvkY3XUiNcXdyF9KzUWVcx+gyGrgivzj21+U+snkaEnR08iAAfIq/eL4UBmy/AFE65Pf/yWQSy32JkzEhnES+p29Y6L8nKTQNXAWUyxS9kuAYTYoabajXtABhtyzeyr0qU51u1OyMHafTWst9frJjgV7nRfhS6ZLy2NLA8b0LLzHKSYgFpWXp0agnRNOepVkKF3hME5D8VETygQGmJ0y6mBZUQB2ImlRYtKccCOc4+fGl4FR6TtNk+6xkqgE=', 'smithi191.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOn05x+aW6hZoPcIlnRTFEqP2h+sLA6MWRf8oZN7upq8Sh1LJtLs8NIBUk77CYZKou7JMWol2TDqqj48SjnrZACgAGmi1oW1suwpw5iYrtRWTIDarYptaFNI/OLDsMjPkEwkkdibeh0rtVRin3J2CAdYud5+m0S9w5dGFfHWouTNsGNuKMkakw/FwMcdDujGCh5QqMeVmbSxV+IZr1w6zeCgthzU4+mDjFpiyaNfoltuvM+2M0OuU0QwM2Osl0hvMCbHRb555xtz66jArweFFcn4PiCX5ati4LidH5bwSJyxMvQjcqPH4xFYa2kX7+84zYLIaW1AH0fiS9oc9dolc0OVsoXlW4qRvAGYkwA+4PB8D76+jPwj//lQpuiU9Q8sZIlHZ79GHwG2+3qLPFGEsglv4eZDP3jt61BKF752g8No+2bBKbWE1HFmH4d/J1E8lMC4HEajx1p3AL5xVmCwAX4Pb++YQ3TOPmyI+c+a++IdygVGlgQPQxpCym+zqTyvc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f6922ca21f63dad2b198f92826f75d268f12fc97'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-24_05:02:42', 'tube': 'smithi', 'user': 'khiremat', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23126'} 2022-12-01T09:46:42.466 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-12-01T09:46:42.469 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-12-01T09:46:42.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/run/ceph 2022-12-01T09:46:42.536 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2022-12-01T09:46:42.582 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:42.582 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2022-12-01T09:46:42.617 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-12-01T09:46:42.617 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2022-12-01T09:46:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-12-01T09:46:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:46:42.674 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 6h/6d Inode: 362101 Links: 1 2022-12-01T09:46:42.674 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:46:42.674 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:46:42.675 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2022-12-01 09:46:33.712149664 +0000 2022-12-01T09:46:42.675 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2022-12-01 09:43:47.962125627 +0000 2022-12-01T09:46:42.675 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2022-12-01 09:43:47.962125627 +0000 2022-12-01T09:46:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2022-12-01T09:46:42.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-12-01T09:46:42.739 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2022-12-01T09:46:42.739 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2022-12-01T09:46:42.740 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000204879 s, 2.5 MB/s 2022-12-01T09:46:42.741 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-12-01T09:46:42.799 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2022-12-01T09:46:42.856 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-12-01T09:46:42.856 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:46:42.857 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 6h/6d Inode: 367804 Links: 1 2022-12-01T09:46:42.857 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:46:42.857 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:46:42.857 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2022-12-01 09:46:33.712149664 +0000 2022-12-01T09:46:42.858 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2022-12-01 09:43:48.492116413 +0000 2022-12-01T09:46:42.858 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2022-12-01 09:43:48.492116413 +0000 2022-12-01T09:46:42.858 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2022-12-01T09:46:42.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-12-01T09:46:42.921 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2022-12-01T09:46:42.921 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2022-12-01T09:46:42.921 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000194433 s, 2.6 MB/s 2022-12-01T09:46:42.922 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-12-01T09:46:42.980 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2022-12-01T09:46:43.039 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-12-01T09:46:43.039 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:46:43.040 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 6h/6d Inode: 364026 Links: 1 2022-12-01T09:46:43.040 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:46:43.040 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:46:43.040 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2022-12-01 09:46:33.712149664 +0000 2022-12-01T09:46:43.041 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2022-12-01 09:43:49.019107252 +0000 2022-12-01T09:46:43.041 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2022-12-01 09:43:49.019107252 +0000 2022-12-01T09:46:43.041 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2022-12-01T09:46:43.041 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-12-01T09:46:43.105 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2022-12-01T09:46:43.106 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2022-12-01T09:46:43.106 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000230542 s, 2.2 MB/s 2022-12-01T09:46:43.107 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-12-01T09:46:43.165 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2022-12-01T09:46:43.222 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-12-01T09:46:43.223 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:46:43.223 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 6h/6d Inode: 366730 Links: 1 2022-12-01T09:46:43.223 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:46:43.223 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:46:43.224 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2022-12-01 09:46:33.712149664 +0000 2022-12-01T09:46:43.224 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2022-12-01 09:43:49.587097378 +0000 2022-12-01T09:46:43.224 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2022-12-01 09:43:49.587097378 +0000 2022-12-01T09:46:43.224 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: - 2022-12-01T09:46:43.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-12-01T09:46:43.288 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2022-12-01T09:46:43.288 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2022-12-01T09:46:43.289 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.00017546 s, 2.9 MB/s 2022-12-01T09:46:43.290 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-12-01T09:46:43.348 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:46:43.349 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:46:43.349 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2022-12-01T09:46:43.371 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-12-01T09:46:43.371 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2022-12-01T09:46:43.431 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-12-01T09:46:43.431 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:46:43.432 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 363998 Links: 1 2022-12-01T09:46:43.432 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:46:43.432 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:46:43.432 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2022-12-01 09:46:38.160979839 +0000 2022-12-01T09:46:43.433 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2022-12-01 09:43:48.503043588 +0000 2022-12-01T09:46:43.433 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2022-12-01 09:43:48.503043588 +0000 2022-12-01T09:46:43.433 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2022-12-01T09:46:43.434 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-12-01T09:46:43.503 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2022-12-01T09:46:43.503 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2022-12-01T09:46:43.504 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000244723 s, 2.1 MB/s 2022-12-01T09:46:43.505 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-12-01T09:46:43.566 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2022-12-01T09:46:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-12-01T09:46:43.625 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:46:43.626 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 360393 Links: 1 2022-12-01T09:46:43.626 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:46:43.626 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:46:43.627 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2022-12-01 09:46:38.160979839 +0000 2022-12-01T09:46:43.627 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2022-12-01 09:43:49.031033850 +0000 2022-12-01T09:46:43.628 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2022-12-01 09:43:49.031033850 +0000 2022-12-01T09:46:43.628 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2022-12-01T09:46:43.628 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-12-01T09:46:43.694 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2022-12-01T09:46:43.694 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2022-12-01T09:46:43.695 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000225681 s, 2.3 MB/s 2022-12-01T09:46:43.696 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-12-01T09:46:43.755 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2022-12-01T09:46:43.814 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-12-01T09:46:43.815 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:46:43.815 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 368790 Links: 1 2022-12-01T09:46:43.815 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:46:43.816 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:46:43.816 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2022-12-01 09:46:38.160979839 +0000 2022-12-01T09:46:43.816 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2022-12-01 09:43:49.588023577 +0000 2022-12-01T09:46:43.816 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2022-12-01 09:43:49.588023577 +0000 2022-12-01T09:46:43.817 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2022-12-01T09:46:43.817 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-12-01T09:46:43.882 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2022-12-01T09:46:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2022-12-01T09:46:43.883 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000220544 s, 2.3 MB/s 2022-12-01T09:46:43.884 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-12-01T09:46:43.943 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2022-12-01T09:46:44.002 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-12-01T09:46:44.003 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-12-01T09:46:44.003 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 359151 Links: 1 2022-12-01T09:46:44.003 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-12-01T09:46:44.003 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2022-12-01T09:46:44.004 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2022-12-01 09:46:38.160979839 +0000 2022-12-01T09:46:44.004 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2022-12-01 09:43:50.136013469 +0000 2022-12-01T09:46:44.004 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2022-12-01 09:43:50.136013469 +0000 2022-12-01T09:46:44.004 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2022-12-01T09:46:44.005 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-12-01T09:46:44.070 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2022-12-01T09:46:44.071 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2022-12-01T09:46:44.071 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000193901 s, 2.6 MB/s 2022-12-01T09:46:44.073 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-12-01T09:46:44.132 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:46:44.132 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-12-01T09:46:44.133 INFO:tasks.ceph:Generating config... 2022-12-01T09:46:44.134 INFO:tasks.ceph:[client] client mount timeout = 600 2022-12-01T09:46:44.135 INFO:tasks.ceph:[client] debug client = 20 2022-12-01T09:46:44.135 INFO:tasks.ceph:[client] debug ms = 1 2022-12-01T09:46:44.136 INFO:tasks.ceph:[client] pid file = 2022-12-01T09:46:44.136 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-12-01T09:46:44.136 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-12-01T09:46:44.136 INFO:tasks.ceph:[mds] debug mds = 20 2022-12-01T09:46:44.137 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-12-01T09:46:44.137 INFO:tasks.ceph:[mds] debug ms = 1 2022-12-01T09:46:44.137 INFO:tasks.ceph:[mds] mds debug frag = True 2022-12-01T09:46:44.138 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-12-01T09:46:44.138 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-12-01T09:46:44.138 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-12-01T09:46:44.139 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-12-01T09:46:44.139 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-12-01T09:46:44.139 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-12-01T09:46:44.140 INFO:tasks.ceph:[mgr] debug client = 20 2022-12-01T09:46:44.140 INFO:tasks.ceph:[mgr] debug finisher = 20 2022-12-01T09:46:44.140 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-12-01T09:46:44.140 INFO:tasks.ceph:[mgr] debug ms = 1 2022-12-01T09:46:44.141 INFO:tasks.ceph:[mon] debug mon = 20 2022-12-01T09:46:44.141 INFO:tasks.ceph:[mon] debug ms = 1 2022-12-01T09:46:44.141 INFO:tasks.ceph:[mon] debug paxos = 20 2022-12-01T09:46:44.142 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-12-01T09:46:44.142 INFO:tasks.ceph:[osd] bdev async discard = True 2022-12-01T09:46:44.142 INFO:tasks.ceph:[osd] bdev enable discard = True 2022-12-01T09:46:44.143 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2022-12-01T09:46:44.143 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-12-01T09:46:44.143 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-12-01T09:46:44.144 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2022-12-01T09:46:44.144 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2022-12-01T09:46:44.144 INFO:tasks.ceph:[osd] debug ms = 1 2022-12-01T09:46:44.145 INFO:tasks.ceph:[osd] debug osd = 20 2022-12-01T09:46:44.145 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2022-12-01T09:46:44.145 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-12-01T09:46:44.145 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-12-01T09:46:44.146 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-12-01T09:46:44.146 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-12-01T09:46:44.146 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-12-01T09:46:44.147 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-12-01T09:46:44.147 INFO:tasks.ceph:Setting up mon.a... 2022-12-01T09:46:44.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-12-01T09:46:44.199 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.keyring 2022-12-01T09:46:44.287 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-12-01T09:46:44.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-12-01T09:46:44.361 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.84'), ('mon.b', '172.21.15.191'), ('mon.c', '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]')] 2022-12-01T09:46:44.362 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.84,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-12-01T09:46:44.364 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:44.364 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-12-01T09:46:44.419 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.84 --add b 172.21.15.191 --addv c '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:46:44.507 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:46:44.508 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: generated fsid 8329cc4d-9cf5-499b-b4c9-a9793681da24 2022-12-01T09:46:44.508 INFO:teuthology.orchestra.run.smithi084.stdout:setting min_mon_release = pacific 2022-12-01T09:46:44.508 INFO:teuthology.orchestra.run.smithi084.stdout:epoch 0 2022-12-01T09:46:44.508 INFO:teuthology.orchestra.run.smithi084.stdout:fsid 8329cc4d-9cf5-499b-b4c9-a9793681da24 2022-12-01T09:46:44.508 INFO:teuthology.orchestra.run.smithi084.stdout:last_changed 2022-12-01T09:46:44.504651+0000 2022-12-01T09:46:44.509 INFO:teuthology.orchestra.run.smithi084.stdout:created 2022-12-01T09:46:44.504651+0000 2022-12-01T09:46:44.509 INFO:teuthology.orchestra.run.smithi084.stdout:min_mon_release 16 (pacific) 2022-12-01T09:46:44.509 INFO:teuthology.orchestra.run.smithi084.stdout:election_strategy: 1 2022-12-01T09:46:44.509 INFO:teuthology.orchestra.run.smithi084.stdout:0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2022-12-01T09:46:44.510 INFO:teuthology.orchestra.run.smithi084.stdout:1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2022-12-01T09:46:44.510 INFO:teuthology.orchestra.run.smithi084.stdout:2: [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] mon.c 2022-12-01T09:46:44.510 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-12-01T09:46:44.512 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-12-01T09:46:44.529 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8329cc4d-9cf5-499b-b4c9-a9793681da24... 2022-12-01T09:46:44.530 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-12-01T09:46:44.571 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-12-01T09:46:44.613 INFO:teuthology.orchestra.run.smithi084.stdout:[global] 2022-12-01T09:46:44.614 INFO:teuthology.orchestra.run.smithi084.stdout: chdir = "" 2022-12-01T09:46:44.614 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-12-01T09:46:44.614 INFO:teuthology.orchestra.run.smithi084.stdout: auth supported = cephx 2022-12-01T09:46:44.614 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.615 INFO:teuthology.orchestra.run.smithi084.stdout: filestore xattr use omap = true 2022-12-01T09:46:44.615 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.615 INFO:teuthology.orchestra.run.smithi084.stdout: mon clock drift allowed = 1.000 2022-12-01T09:46:44.615 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.616 INFO:teuthology.orchestra.run.smithi084.stdout: osd crush chooseleaf type = 0 2022-12-01T09:46:44.616 INFO:teuthology.orchestra.run.smithi084.stdout: auth debug = true 2022-12-01T09:46:44.616 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.616 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on old message = true 2022-12-01T09:46:44.617 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on bug = true 2022-12-01T09:46:44.617 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.617 INFO:teuthology.orchestra.run.smithi084.stdout: mon max pg per osd = 10000 # >= luminous 2022-12-01T09:46:44.617 INFO:teuthology.orchestra.run.smithi084.stdout: mon pg warn max object skew = 0 2022-12-01T09:46:44.618 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.618 INFO:teuthology.orchestra.run.smithi084.stdout: # disable pg_autoscaler by default for new pools 2022-12-01T09:46:44.618 INFO:teuthology.orchestra.run.smithi084.stdout: osd_pool_default_pg_autoscale_mode = off 2022-12-01T09:46:44.618 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.619 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default size = 2 2022-12-01T09:46:44.619 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.619 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow primary affinity = true 2022-12-01T09:46:44.619 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow pg remap = true 2022-12-01T09:46:44.620 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on legacy crush tunables = false 2022-12-01T09:46:44.620 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on crush straw calc version zero = false 2022-12-01T09:46:44.620 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on no sortbitwise = false 2022-12-01T09:46:44.620 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on osd down out interval zero = false 2022-12-01T09:46:44.620 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on too few osds = false 2022-12-01T09:46:44.621 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-12-01T09:46:44.621 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_no_redundancy = false 2022-12-01T09:46:44.621 INFO:teuthology.orchestra.run.smithi084.stdout: mon_allow_pool_size_one = true 2022-12-01T09:46:44.621 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.622 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-12-01T09:46:44.622 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.622 INFO:teuthology.orchestra.run.smithi084.stdout: osd default data pool replay window = 5 2022-12-01T09:46:44.622 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.623 INFO:teuthology.orchestra.run.smithi084.stdout: mon allow pool delete = true 2022-12-01T09:46:44.623 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.623 INFO:teuthology.orchestra.run.smithi084.stdout: mon cluster log file level = debug 2022-12-01T09:46:44.623 INFO:teuthology.orchestra.run.smithi084.stdout: debug asserts on shutdown = true 2022-12-01T09:46:44.623 INFO:teuthology.orchestra.run.smithi084.stdout: mon health detail to clog = false 2022-12-01T09:46:44.624 INFO:teuthology.orchestra.run.smithi084.stdout: mon host = "172.21.15.84,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2022-12-01T09:46:44.624 INFO:teuthology.orchestra.run.smithi084.stdout: fsid = 8329cc4d-9cf5-499b-b4c9-a9793681da24 2022-12-01T09:46:44.624 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.624 INFO:teuthology.orchestra.run.smithi084.stdout:[osd] 2022-12-01T09:46:44.625 INFO:teuthology.orchestra.run.smithi084.stdout: osd journal size = 100 2022-12-01T09:46:44.625 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.625 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub load threshold = 5.0 2022-12-01T09:46:44.625 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub max interval = 600 2022-12-01T09:46:44.626 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.626 INFO:teuthology.orchestra.run.smithi084.stdout: osd recover clone overlap = true 2022-12-01T09:46:44.626 INFO:teuthology.orchestra.run.smithi084.stdout: osd recovery max chunk = 1048576 2022-12-01T09:46:44.626 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.627 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug shutdown = true 2022-12-01T09:46:44.627 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug op order = true 2022-12-01T09:46:44.627 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug verify stray on activate = true 2022-12-01T09:46:44.627 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.628 INFO:teuthology.orchestra.run.smithi084.stdout: osd open classes on start = true 2022-12-01T09:46:44.628 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug pg log writeout = true 2022-12-01T09:46:44.628 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.628 INFO:teuthology.orchestra.run.smithi084.stdout: osd deep scrub update digest min age = 30 2022-12-01T09:46:44.628 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.629 INFO:teuthology.orchestra.run.smithi084.stdout: osd map max advance = 10 2022-12-01T09:46:44.629 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.629 INFO:teuthology.orchestra.run.smithi084.stdout: journal zero on create = true 2022-12-01T09:46:44.629 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.629 INFO:teuthology.orchestra.run.smithi084.stdout: filestore ondisk finisher threads = 3 2022-12-01T09:46:44.630 INFO:teuthology.orchestra.run.smithi084.stdout: filestore apply finisher threads = 3 2022-12-01T09:46:44.630 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.630 INFO:teuthology.orchestra.run.smithi084.stdout: bdev debug aio = true 2022-12-01T09:46:44.630 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug misdirected ops = true 2022-12-01T09:46:44.631 INFO:teuthology.orchestra.run.smithi084.stdout: bdev async discard = True 2022-12-01T09:46:44.631 INFO:teuthology.orchestra.run.smithi084.stdout: bdev enable discard = True 2022-12-01T09:46:44.631 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore allocator = bitmap 2022-12-01T09:46:44.631 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore block size = 96636764160 2022-12-01T09:46:44.632 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore fsck on mount = True 2022-12-01T09:46:44.632 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluefs = 1/20 2022-12-01T09:46:44.632 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluestore = 1/20 2022-12-01T09:46:44.632 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2022-12-01T09:46:44.633 INFO:teuthology.orchestra.run.smithi084.stdout: debug osd = 20 2022-12-01T09:46:44.633 INFO:teuthology.orchestra.run.smithi084.stdout: debug rocksdb = 4/10 2022-12-01T09:46:44.633 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd backfillfull_ratio = 0.85 2022-12-01T09:46:44.633 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd full ratio = 0.9 2022-12-01T09:46:44.634 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd nearfull ratio = 0.8 2022-12-01T09:46:44.634 INFO:teuthology.orchestra.run.smithi084.stdout: osd failsafe full ratio = 0.95 2022-12-01T09:46:44.634 INFO:teuthology.orchestra.run.smithi084.stdout: osd objectstore = bluestore 2022-12-01T09:46:44.634 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2022-12-01T09:46:44.634 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.635 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr] 2022-12-01T09:46:44.635 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2022-12-01T09:46:44.635 INFO:teuthology.orchestra.run.smithi084.stdout: debug mgr = 20 2022-12-01T09:46:44.635 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2022-12-01T09:46:44.636 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2022-12-01T09:46:44.636 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2022-12-01T09:46:44.636 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2022-12-01T09:46:44.636 INFO:teuthology.orchestra.run.smithi084.stdout: mgr/telemetry/nag = false 2022-12-01T09:46:44.637 INFO:teuthology.orchestra.run.smithi084.stdout: debug client = 20 2022-12-01T09:46:44.637 INFO:teuthology.orchestra.run.smithi084.stdout: debug finisher = 20 2022-12-01T09:46:44.637 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.637 INFO:teuthology.orchestra.run.smithi084.stdout:[mon] 2022-12-01T09:46:44.638 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2022-12-01T09:46:44.638 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2022-12-01T09:46:44.638 INFO:teuthology.orchestra.run.smithi084.stdout: debug paxos = 20 2022-12-01T09:46:44.638 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2022-12-01T09:46:44.638 INFO:teuthology.orchestra.run.smithi084.stdout: mon data avail warn = 5 2022-12-01T09:46:44.639 INFO:teuthology.orchestra.run.smithi084.stdout: mon mgr mkfs grace = 240 2022-12-01T09:46:44.639 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2022-12-01T09:46:44.639 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd reporter subtree level = osd 2022-12-01T09:46:44.639 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd prime pg temp = true 2022-12-01T09:46:44.640 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2022-12-01T09:46:44.640 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.640 INFO:teuthology.orchestra.run.smithi084.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-12-01T09:46:44.640 INFO:teuthology.orchestra.run.smithi084.stdout: auth mon ticket ttl = 660 # 11m 2022-12-01T09:46:44.641 INFO:teuthology.orchestra.run.smithi084.stdout: auth service ticket ttl = 240 # 4m 2022-12-01T09:46:44.641 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.641 INFO:teuthology.orchestra.run.smithi084.stdout: # don't complain about insecure global_id in the test suite 2022-12-01T09:46:44.641 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-12-01T09:46:44.642 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-12-01T09:46:44.642 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.642 INFO:teuthology.orchestra.run.smithi084.stdout: # 1m isn't quite enough 2022-12-01T09:46:44.642 INFO:teuthology.orchestra.run.smithi084.stdout: mon_down_mkfs_grace = 2m 2022-12-01T09:46:44.643 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.643 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_filestore_osds = false 2022-12-01T09:46:44.643 INFO:teuthology.orchestra.run.smithi084.stdout: mon op complaint time = 120 2022-12-01T09:46:44.643 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:46:44.643 INFO:teuthology.orchestra.run.smithi084.stdout:[client] 2022-12-01T09:46:44.644 INFO:teuthology.orchestra.run.smithi084.stdout: rgw cache enabled = true 2022-12-01T09:46:44.644 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable ops log = true 2022-12-01T09:46:44.644 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable usage log = true 2022-12-01T09:46:44.644 INFO:teuthology.orchestra.run.smithi084.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-12-01T09:46:44.645 INFO:teuthology.orchestra.run.smithi084.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-12-01T09:46:44.645 INFO:teuthology.orchestra.run.smithi084.stdout: client mount timeout = 600 2022-12-01T09:46:44.645 INFO:teuthology.orchestra.run.smithi084.stdout: debug client = 20 2022-12-01T09:46:44.645 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2022-12-01T09:46:44.646 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = "" 2022-12-01T09:46:44.646 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2022-12-01T09:46:44.646 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2022-12-01T09:46:44.646 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.a] 2022-12-01T09:46:44.647 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.b] 2022-12-01T09:46:44.647 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.c] 2022-12-01T09:46:44.647 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2022-12-01T09:46:44.647 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2022-12-01T09:46:44.647 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2022-12-01T09:46:44.648 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2022-12-01T09:46:44.648 INFO:teuthology.orchestra.run.smithi084.stdout:[mds] 2022-12-01T09:46:44.648 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds = 20 2022-12-01T09:46:44.648 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds balancer = 20 2022-12-01T09:46:44.649 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2022-12-01T09:46:44.649 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug frag = True 2022-12-01T09:46:44.649 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug scatterstat = True 2022-12-01T09:46:44.649 INFO:teuthology.orchestra.run.smithi084.stdout: mds op complaint time = 180 2022-12-01T09:46:44.650 INFO:teuthology.orchestra.run.smithi084.stdout: mds verify scatter = True 2022-12-01T09:46:44.650 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2022-12-01T09:46:44.650 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2022-12-01T09:46:44.650 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2022-12-01T09:46:44.652 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2022-12-01T09:46:44.653 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2022-12-01T09:46:44.653 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-12-01T09:46:44.653 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2022-12-01T09:46:44.653 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.654 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2022-12-01T09:46:44.654 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.654 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2022-12-01T09:46:44.654 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.654 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2022-12-01T09:46:44.655 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2022-12-01T09:46:44.655 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.655 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2022-12-01T09:46:44.655 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2022-12-01T09:46:44.656 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.656 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2022-12-01T09:46:44.656 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2022-12-01T09:46:44.656 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.657 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2022-12-01T09:46:44.657 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2022-12-01T09:46:44.657 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.657 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2022-12-01T09:46:44.657 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.658 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2022-12-01T09:46:44.658 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2022-12-01T09:46:44.658 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2022-12-01T09:46:44.658 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2022-12-01T09:46:44.659 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2022-12-01T09:46:44.659 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2022-12-01T09:46:44.659 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2022-12-01T09:46:44.659 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-12-01T09:46:44.660 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2022-12-01T09:46:44.660 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2022-12-01T09:46:44.660 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.660 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-12-01T09:46:44.660 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.661 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2022-12-01T09:46:44.661 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.661 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2022-12-01T09:46:44.661 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.662 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2022-12-01T09:46:44.662 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2022-12-01T09:46:44.662 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2022-12-01T09:46:44.662 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.84,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2022-12-01T09:46:44.663 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = 8329cc4d-9cf5-499b-b4c9-a9793681da24 2022-12-01T09:46:44.663 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.663 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2022-12-01T09:46:44.663 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2022-12-01T09:46:44.664 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.664 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2022-12-01T09:46:44.664 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2022-12-01T09:46:44.664 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.664 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2022-12-01T09:46:44.665 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2022-12-01T09:46:44.665 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.665 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2022-12-01T09:46:44.665 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2022-12-01T09:46:44.666 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2022-12-01T09:46:44.666 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.666 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2022-12-01T09:46:44.666 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2022-12-01T09:46:44.667 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.667 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2022-12-01T09:46:44.667 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.667 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2022-12-01T09:46:44.667 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.668 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2022-12-01T09:46:44.668 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.668 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2022-12-01T09:46:44.668 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2022-12-01T09:46:44.669 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.669 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2022-12-01T09:46:44.669 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2022-12-01T09:46:44.669 INFO:teuthology.orchestra.run.smithi191.stdout: bdev async discard = True 2022-12-01T09:46:44.670 INFO:teuthology.orchestra.run.smithi191.stdout: bdev enable discard = True 2022-12-01T09:46:44.670 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore allocator = bitmap 2022-12-01T09:46:44.670 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2022-12-01T09:46:44.670 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2022-12-01T09:46:44.670 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 1/20 2022-12-01T09:46:44.671 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 1/20 2022-12-01T09:46:44.671 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-12-01T09:46:44.671 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2022-12-01T09:46:44.671 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 4/10 2022-12-01T09:46:44.672 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2022-12-01T09:46:44.672 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2022-12-01T09:46:44.672 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2022-12-01T09:46:44.672 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 0.95 2022-12-01T09:46:44.673 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = bluestore 2022-12-01T09:46:44.673 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-12-01T09:46:44.673 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.673 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2022-12-01T09:46:44.673 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-12-01T09:46:44.674 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2022-12-01T09:46:44.674 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-12-01T09:46:44.674 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-12-01T09:46:44.674 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-12-01T09:46:44.675 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-12-01T09:46:44.675 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2022-12-01T09:46:44.675 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2022-12-01T09:46:44.675 INFO:teuthology.orchestra.run.smithi191.stdout: debug finisher = 20 2022-12-01T09:46:44.676 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.676 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2022-12-01T09:46:44.676 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-12-01T09:46:44.676 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2022-12-01T09:46:44.677 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2022-12-01T09:46:44.677 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2022-12-01T09:46:44.677 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2022-12-01T09:46:44.677 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2022-12-01T09:46:44.677 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2022-12-01T09:46:44.678 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2022-12-01T09:46:44.678 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2022-12-01T09:46:44.678 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2022-12-01T09:46:44.678 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.679 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-12-01T09:46:44.679 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2022-12-01T09:46:44.679 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2022-12-01T09:46:44.679 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.679 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2022-12-01T09:46:44.680 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-12-01T09:46:44.680 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-12-01T09:46:44.680 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.680 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2022-12-01T09:46:44.681 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2022-12-01T09:46:44.681 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.681 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2022-12-01T09:46:44.681 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2022-12-01T09:46:44.682 INFO:teuthology.orchestra.run.smithi191.stdout: 2022-12-01T09:46:44.682 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2022-12-01T09:46:44.682 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2022-12-01T09:46:44.682 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2022-12-01T09:46:44.682 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2022-12-01T09:46:44.683 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-12-01T09:46:44.683 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-12-01T09:46:44.683 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2022-12-01T09:46:44.683 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2022-12-01T09:46:44.684 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-12-01T09:46:44.684 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = "" 2022-12-01T09:46:44.684 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2022-12-01T09:46:44.684 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2022-12-01T09:46:44.685 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2022-12-01T09:46:44.685 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2022-12-01T09:46:44.685 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2022-12-01T09:46:44.685 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2022-12-01T09:46:44.685 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2022-12-01T09:46:44.686 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2022-12-01T09:46:44.686 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2022-12-01T09:46:44.686 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2022-12-01T09:46:44.686 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2022-12-01T09:46:44.687 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds balancer = 20 2022-12-01T09:46:44.687 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2022-12-01T09:46:44.687 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2022-12-01T09:46:44.687 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2022-12-01T09:46:44.688 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2022-12-01T09:46:44.688 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2022-12-01T09:46:44.688 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2022-12-01T09:46:44.688 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2022-12-01T09:46:44.689 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2022-12-01T09:46:44.689 INFO:tasks.ceph:Creating admin key on mon.a... 2022-12-01T09:46:44.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-12-01T09:46:44.748 INFO:tasks.ceph:Copying monmap to all nodes... 2022-12-01T09:46:44.749 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:44.749 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-12-01T09:46:44.766 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:44.767 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-12-01T09:46:44.823 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi084.front.sepia.ceph.com 2022-12-01T09:46:44.823 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:44.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.keyring 2022-12-01T09:46:44.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-12-01T09:46:44.903 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:46:44.904 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:46:45.043 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2022-12-01T09:46:45.043 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:46:45.044 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2022-12-01T09:46:45.044 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-12-01T09:46:45.087 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:46:45.088 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:46:45.147 INFO:tasks.ceph:Setting up mon nodes... 2022-12-01T09:46:45.148 INFO:tasks.ceph:Setting up mgr nodes... 2022-12-01T09:46:45.149 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-12-01T09:46:45.202 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-12-01T09:46:45.205 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-12-01T09:46:45.275 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-12-01T09:46:45.278 INFO:tasks.ceph:Setting up mds nodes... 2022-12-01T09:46:45.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-12-01T09:46:45.332 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-12-01T09:46:45.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-12-01T09:46:45.361 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-12-01T09:46:45.454 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-12-01T09:46:45.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-12-01T09:46:45.483 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-12-01T09:46:45.545 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-12-01T09:46:45.548 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-12-01T09:46:45.576 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-12-01T09:46:45.673 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-12-01T09:46:45.676 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-12-01T09:46:45.703 INFO:tasks.ceph_client:Setting up client nodes... 2022-12-01T09:46:45.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-12-01T09:46:45.748 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-12-01T09:46:45.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-12-01T09:46:45.804 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-12-01T09:46:45.816 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-12-01T09:46:45.861 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-12-01T09:46:45.874 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2022-12-01T09:46:45.917 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.3.keyring 2022-12-01T09:46:45.930 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-12-01T09:46:45.930 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-12-01T09:46:45.931 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-12-01T09:46:45.958 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:46:45.959 INFO:tasks.ceph:role: osd.0 2022-12-01T09:46:45.959 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com 2022-12-01T09:46:45.960 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-12-01T09:46:46.032 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:46:46.033 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:46:46.033 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:46:46.034 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:46:46.034 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:46:46.034 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:46:46.034 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:46:46.035 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:46:46.035 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:46:46.035 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:46:46.454 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2022-12-01T09:46:46.456 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2022-12-01T09:46:46.456 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-12-01T09:46:46.594 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-12-01T09:46:46.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-12-01T09:46:46.696 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:46:46.697 INFO:tasks.ceph:role: osd.1 2022-12-01T09:46:46.698 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com 2022-12-01T09:46:46.698 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-12-01T09:46:46.770 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:46:46.771 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:46:46.771 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:46:46.771 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:46:46.771 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:46:46.772 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:46:46.772 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:46:46.772 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:46:46.772 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:46:46.773 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:46:47.204 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2022-12-01T09:46:47.205 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2022-12-01T09:46:47.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-12-01T09:46:47.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-12-01T09:46:47.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-12-01T09:46:47.390 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:46:47.391 INFO:tasks.ceph:role: osd.2 2022-12-01T09:46:47.391 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com 2022-12-01T09:46:47.392 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-12-01T09:46:47.464 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:46:47.464 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:46:47.465 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:46:47.465 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:46:47.465 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:46:47.465 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:46:47.466 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:46:47.466 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:46:47.466 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:46:47.466 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:46:47.908 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2022-12-01T09:46:47.910 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2022-12-01T09:46:47.910 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-12-01T09:46:48.021 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-12-01T09:46:48.053 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-12-01T09:46:48.121 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-12-01T09:46:48.121 INFO:tasks.ceph:role: osd.3 2022-12-01T09:46:48.122 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com 2022-12-01T09:46:48.122 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-12-01T09:46:48.195 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:46:48.195 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:46:48.196 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:46:48.196 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:46:48.196 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:46:48.197 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:46:48.197 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:46:48.197 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:46:48.197 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:46:48.198 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:46:48.632 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2022-12-01T09:46:48.634 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2022-12-01T09:46:48.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-12-01T09:46:48.799 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-12-01T09:46:48.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:46:48.920 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:48.916+0000 7ff8d2e8f5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-12-01T09:46:48.923 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:48.919+0000 7ff8d2e8f5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-12-01T09:46:48.923 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:48.919+0000 7ff8d2e8f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-12-01T09:46:48.924 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:48.919+0000 7ff8d2e8f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-12-01T09:46:48.924 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:48.919+0000 7ff8d2e8f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-12-01T09:46:48.928 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:48.924+0000 7ff8d2e8f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-12-01T09:46:51.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-12-01T09:46:51.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:46:51.924 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:51.918+0000 7f79690b25c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-12-01T09:46:51.925 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:51.919+0000 7f79690b25c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-12-01T09:46:51.925 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:51.919+0000 7f79690b25c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-12-01T09:46:51.926 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:51.919+0000 7f79690b25c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-12-01T09:46:51.926 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:51.919+0000 7f79690b25c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-12-01T09:46:51.927 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:51.919+0000 7f79690b25c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-12-01T09:46:54.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-12-01T09:46:54.822 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:46:54.914 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:54.909+0000 7f72c4b0c5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-12-01T09:46:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:54.910+0000 7f72c4b0c5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-12-01T09:46:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:54.910+0000 7f72c4b0c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-12-01T09:46:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:54.910+0000 7f72c4b0c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-12-01T09:46:54.915 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:54.910+0000 7f72c4b0c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-12-01T09:46:54.916 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:54.910+0000 7f72c4b0c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-12-01T09:46:57.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-12-01T09:46:57.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:46:57.896 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:57.891+0000 7ff0e4ddc5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-12-01T09:46:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:57.892+0000 7ff0e4ddc5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-12-01T09:46:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:57.892+0000 7ff0e4ddc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-12-01T09:46:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:57.892+0000 7ff0e4ddc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-12-01T09:46:57.897 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:57.892+0000 7ff0e4ddc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-12-01T09:46:57.898 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:46:57.892+0000 7ff0e4ddc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-12-01T09:47:00.754 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-12-01T09:47:00.786 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-12-01T09:47:00.819 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:47:00.819 INFO:tasks.ceph:role: osd.4 2022-12-01T09:47:00.820 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2022-12-01T09:47:00.820 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-12-01T09:47:00.897 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:47:00.898 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:47:00.899 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:47:00.899 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:47:00.900 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:47:00.900 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:47:00.900 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:47:00.901 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:47:00.901 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:47:00.902 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:47:01.322 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2022-12-01T09:47:01.324 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2022-12-01T09:47:01.325 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-12-01T09:47:01.478 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-12-01T09:47:01.517 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-12-01T09:47:01.586 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:47:01.586 INFO:tasks.ceph:role: osd.5 2022-12-01T09:47:01.587 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2022-12-01T09:47:01.588 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-12-01T09:47:01.661 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:47:01.661 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:47:01.661 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:47:01.662 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:47:01.662 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:47:01.662 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:47:01.662 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:47:01.663 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:47:01.663 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:47:01.663 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:47:02.089 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2022-12-01T09:47:02.091 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2022-12-01T09:47:02.092 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-12-01T09:47:02.212 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-12-01T09:47:02.246 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-12-01T09:47:02.318 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:47:02.319 INFO:tasks.ceph:role: osd.6 2022-12-01T09:47:02.320 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2022-12-01T09:47:02.320 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-12-01T09:47:02.395 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:47:02.395 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:47:02.396 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:47:02.396 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:47:02.396 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:47:02.397 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:47:02.397 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:47:02.398 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:47:02.398 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:47:02.398 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:47:02.851 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2022-12-01T09:47:02.853 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2022-12-01T09:47:02.853 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-12-01T09:47:02.999 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-12-01T09:47:03.035 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-12-01T09:47:03.103 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-12-01T09:47:03.104 INFO:tasks.ceph:role: osd.7 2022-12-01T09:47:03.104 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com 2022-12-01T09:47:03.105 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-12-01T09:47:03.178 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-12-01T09:47:03.179 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2022-12-01T09:47:03.179 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-12-01T09:47:03.180 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-12-01T09:47:03.180 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-12-01T09:47:03.181 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2022-12-01T09:47:03.181 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-12-01T09:47:03.181 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-12-01T09:47:03.181 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-12-01T09:47:03.182 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-12-01T09:47:03.613 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2022-12-01T09:47:03.615 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2022-12-01T09:47:03.616 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-12-01T09:47:03.781 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-12-01T09:47:03.816 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:47:03.903 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:03.898+0000 7f5e5293f5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-12-01T09:47:03.903 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:03.898+0000 7f5e5293f5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-12-01T09:47:03.904 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:03.898+0000 7f5e5293f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-12-01T09:47:03.904 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:03.898+0000 7f5e5293f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-12-01T09:47:03.904 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:03.898+0000 7f5e5293f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-12-01T09:47:03.904 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:03.899+0000 7f5e5293f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-12-01T09:47:06.778 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-12-01T09:47:06.810 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:47:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:06.897+0000 7fc1b26495c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-12-01T09:47:06.902 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:06.897+0000 7fc1b26495c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-12-01T09:47:06.903 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:06.897+0000 7fc1b26495c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-12-01T09:47:06.903 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:06.897+0000 7fc1b26495c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-12-01T09:47:06.903 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:06.897+0000 7fc1b26495c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-12-01T09:47:06.904 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:06.898+0000 7fc1b26495c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-12-01T09:47:09.763 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-12-01T09:47:09.796 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:47:09.889 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:09.884+0000 7f0cc03865c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-12-01T09:47:09.889 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:09.884+0000 7f0cc03865c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-12-01T09:47:09.889 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:09.884+0000 7f0cc03865c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-12-01T09:47:09.890 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:09.884+0000 7f0cc03865c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-12-01T09:47:09.890 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:09.884+0000 7f0cc03865c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-12-01T09:47:09.890 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:09.885+0000 7f0cc03865c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-12-01T09:47:12.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-12-01T09:47:12.472 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-12-01T09:47:12.745 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-12-01T09:47:12.776 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:47:12.867 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:12.863+0000 7fca6f0125c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-12-01T09:47:12.867 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:12.863+0000 7fca6f0125c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-12-01T09:47:12.868 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:12.863+0000 7fca6f0125c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-12-01T09:47:12.868 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:12.863+0000 7fca6f0125c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-12-01T09:47:12.869 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:12.863+0000 7fca6f0125c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-12-01T09:47:12.869 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:12.864+0000 7fca6f0125c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-12-01T09:47:15.734 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-12-01T09:47:15.767 INFO:tasks.ceph:Reading keys from all nodes... 2022-12-01T09:47:15.768 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:15.768 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-12-01T09:47:15.803 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:15.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-12-01T09:47:15.873 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:15.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-12-01T09:47:15.944 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:15.945 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-12-01T09:47:16.014 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:16.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-12-01T09:47:16.085 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:16.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-12-01T09:47:16.154 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:16.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-12-01T09:47:16.223 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.224 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-12-01T09:47:16.256 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.256 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-12-01T09:47:16.327 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.328 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-12-01T09:47:16.397 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.397 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-12-01T09:47:16.466 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.467 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-12-01T09:47:16.535 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.535 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-12-01T09:47:16.603 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.604 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-12-01T09:47:16.673 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:16.674 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-12-01T09:47:16.695 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:16.696 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2022-12-01T09:47:16.756 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.757 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-12-01T09:47:16.776 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:16.777 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2022-12-01T09:47:16.833 INFO:tasks.ceph:Adding keys to all mons... 2022-12-01T09:47:16.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo tee -a /etc/ceph/ceph.keyring 2022-12-01T09:47:16.836 DEBUG:teuthology.orchestra.run.smithi191:> sudo tee -a /etc/ceph/ceph.keyring 2022-12-01T09:47:16.876 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.y] 2022-12-01T09:47:16.877 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhjt1vtCxAA6fIq6WfkCtSTkz0HLEbDGA== 2022-12-01T09:47:16.877 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2022-12-01T09:47:16.878 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhjgO6oExAACnwQIp+4lfSYzw6avqwsAw== 2022-12-01T09:47:16.878 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2022-12-01T09:47:16.878 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhjTa70GhAAeMPDYcU5YesCdkSQwBnU2w== 2022-12-01T09:47:16.878 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.0] 2022-12-01T09:47:16.879 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAIeIhjAiuxNhAAuFLMZib7/PXEJavlFdcD6w== 2022-12-01T09:47:16.879 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.1] 2022-12-01T09:47:16.879 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQALeIhjppDYNhAAtNoYEkrmkv/Kqd439CfKrg== 2022-12-01T09:47:16.880 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.2] 2022-12-01T09:47:16.880 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAOeIhjgKRPNhAAGJ0EXpQB66+Nhs6mgd+Ryg== 2022-12-01T09:47:16.880 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.3] 2022-12-01T09:47:16.880 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAReIhjay4+NRAA+pjIOF1GNBs2czrdRKaLeQ== 2022-12-01T09:47:16.880 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.x] 2022-12-01T09:47:16.881 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhje+FEEBAAIK2Xgy2xaAwHl9kPA7HcdA== 2022-12-01T09:47:16.881 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2022-12-01T09:47:16.881 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhjNPVVIBAAJ74pkD/1j1u24XFM3WIScg== 2022-12-01T09:47:16.881 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2022-12-01T09:47:16.882 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhjrjD+JxAAxp7Eh6qasyENOB1BscBeag== 2022-12-01T09:47:16.882 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.4] 2022-12-01T09:47:16.882 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAXeIhjb3mjNRAAI9dLHuEE5v3gCFrk87yrUg== 2022-12-01T09:47:16.882 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.5] 2022-12-01T09:47:16.882 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAaeIhjjFeVNRAAXeLN7HIr4BkGOd0sMJMpnQ== 2022-12-01T09:47:16.883 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.6] 2022-12-01T09:47:16.883 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAdeIhj2IPLNBAAdORCPEyy3hD9sQZvsPJrXg== 2022-12-01T09:47:16.883 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.7] 2022-12-01T09:47:16.883 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAgeIhj6D19MxAAnZJuLMl+GtFUhi4Ap4G8+Q== 2022-12-01T09:47:16.884 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2022-12-01T09:47:16.884 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhjtbpsLBAAt3yZts7FbIsq4z6gC305+Q== 2022-12-01T09:47:16.884 INFO:teuthology.orchestra.run.smithi084.stdout:[client.2] 2022-12-01T09:47:16.884 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhjTm7BLxAAJWw+rf/VukYRa9sHgROw/A== 2022-12-01T09:47:16.885 INFO:teuthology.orchestra.run.smithi084.stdout:[client.1] 2022-12-01T09:47:16.885 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhj8W4zMxAA/YcxvdyRPh6RQgiRylkcpA== 2022-12-01T09:47:16.885 INFO:teuthology.orchestra.run.smithi084.stdout:[client.3] 2022-12-01T09:47:16.885 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQAFeIhjpj6BNhAAWDLBNmm08UJdzw64DWgL8A== 2022-12-01T09:47:16.898 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.y] 2022-12-01T09:47:16.899 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhjt1vtCxAA6fIq6WfkCtSTkz0HLEbDGA== 2022-12-01T09:47:16.899 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2022-12-01T09:47:16.899 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhjgO6oExAACnwQIp+4lfSYzw6avqwsAw== 2022-12-01T09:47:16.899 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2022-12-01T09:47:16.900 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhjTa70GhAAeMPDYcU5YesCdkSQwBnU2w== 2022-12-01T09:47:16.900 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.0] 2022-12-01T09:47:16.900 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAIeIhjAiuxNhAAuFLMZib7/PXEJavlFdcD6w== 2022-12-01T09:47:16.900 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.1] 2022-12-01T09:47:16.901 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQALeIhjppDYNhAAtNoYEkrmkv/Kqd439CfKrg== 2022-12-01T09:47:16.901 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.2] 2022-12-01T09:47:16.901 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAOeIhjgKRPNhAAGJ0EXpQB66+Nhs6mgd+Ryg== 2022-12-01T09:47:16.901 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.3] 2022-12-01T09:47:16.902 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAReIhjay4+NRAA+pjIOF1GNBs2czrdRKaLeQ== 2022-12-01T09:47:16.902 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.x] 2022-12-01T09:47:16.902 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhje+FEEBAAIK2Xgy2xaAwHl9kPA7HcdA== 2022-12-01T09:47:16.902 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2022-12-01T09:47:16.902 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhjNPVVIBAAJ74pkD/1j1u24XFM3WIScg== 2022-12-01T09:47:16.903 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2022-12-01T09:47:16.903 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhjrjD+JxAAxp7Eh6qasyENOB1BscBeag== 2022-12-01T09:47:16.903 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.4] 2022-12-01T09:47:16.903 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAXeIhjb3mjNRAAI9dLHuEE5v3gCFrk87yrUg== 2022-12-01T09:47:16.904 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.5] 2022-12-01T09:47:16.904 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAaeIhjjFeVNRAAXeLN7HIr4BkGOd0sMJMpnQ== 2022-12-01T09:47:16.904 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.6] 2022-12-01T09:47:16.904 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAdeIhj2IPLNBAAdORCPEyy3hD9sQZvsPJrXg== 2022-12-01T09:47:16.904 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.7] 2022-12-01T09:47:16.905 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAgeIhj6D19MxAAnZJuLMl+GtFUhi4Ap4G8+Q== 2022-12-01T09:47:16.905 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2022-12-01T09:47:16.905 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhjtbpsLBAAt3yZts7FbIsq4z6gC305+Q== 2022-12-01T09:47:16.906 INFO:teuthology.orchestra.run.smithi191.stdout:[client.2] 2022-12-01T09:47:16.906 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhjTm7BLxAAJWw+rf/VukYRa9sHgROw/A== 2022-12-01T09:47:16.906 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2022-12-01T09:47:16.906 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhj8W4zMxAA/YcxvdyRPh6RQgiRylkcpA== 2022-12-01T09:47:16.906 INFO:teuthology.orchestra.run.smithi191.stdout:[client.3] 2022-12-01T09:47:16.907 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQAFeIhjpj6BNhAAWDLBNmm08UJdzw64DWgL8A== 2022-12-01T09:47:16.908 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-12-01T09:47:16.929 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-12-01T09:47:17.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:47:17.005 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:47:17.063 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:47:17.065 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:47:17.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.124 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.183 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.236 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.239 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.294 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.348 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-12-01T09:47:17.351 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-12-01T09:47:17.404 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:47:17.407 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:47:17.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:47:17.466 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-12-01T09:47:17.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.525 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.592 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.676 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.729 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.732 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-12-01T09:47:17.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:47:17.787 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:47:17.844 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:47:17.846 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:47:17.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:47:17.904 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:47:17.960 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:47:17.963 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-12-01T09:47:18.019 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-12-01T09:47:18.019 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-12-01T09:47:18.047 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-12-01T09:47:18.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-12-01T09:47:18.172 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-12-01T09:47:18.199 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-12-01T09:47:18.296 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-12-01T09:47:18.323 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-12-01T09:47:18.390 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-12-01T09:47:18.488 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-12-01T09:47:18.516 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:47:18.519 DEBUG:teuthology.orchestra.run.smithi191:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-12-01T09:47:18.572 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-12-01T09:47:18.574 INFO:tasks.ceph.mon.a:Restarting daemon 2022-12-01T09:47:18.574 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-12-01T09:47:18.577 INFO:tasks.ceph.mon.a:Started 2022-12-01T09:47:18.577 INFO:tasks.ceph.mon.b:Restarting daemon 2022-12-01T09:47:18.578 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-12-01T09:47:18.615 INFO:tasks.ceph.mon.b:Started 2022-12-01T09:47:18.616 INFO:tasks.ceph.mon.c:Restarting daemon 2022-12-01T09:47:18.616 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-12-01T09:47:18.618 INFO:tasks.ceph.mon.c:Started 2022-12-01T09:47:18.619 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-12-01T09:47:18.619 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-12-01T09:47:18.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-12-01T09:47:18.622 INFO:tasks.ceph.mgr.y:Started 2022-12-01T09:47:18.623 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-12-01T09:47:18.623 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-12-01T09:47:18.625 INFO:tasks.ceph.mgr.x:Started 2022-12-01T09:47:18.626 INFO:tasks.ceph:Setting crush tunables to default 2022-12-01T09:47:18.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd crush tunables default 2022-12-01T09:47:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.744+0000 7fd745d6b700 1 Processor -- start 2022-12-01T09:47:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.744+0000 7fd745d6b700 1 -- start start 2022-12-01T09:47:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd745d6b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd745d6b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd745d6b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74006f7e0 con 0x7fd74006ffc0 2022-12-01T09:47:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd745d6b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7400708b0 con 0x7fd740070430 2022-12-01T09:47:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd745d6b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400cd640 con 0x7fd74015b810 2022-12-01T09:47:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd73ffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd73f7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd73ffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.84:57564/0 (socket says 172.21.15.84:57564) 2022-12-01T09:47:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 learned_addr learned my addr 172.21.15.84:0/899444440 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.745+0000 7fd73effd700 1 --1- 172.21.15.84:0/899444440 >> v1:172.21.15.191:6789/0 conn(0x7fd74006ffc0 0x7fd74006add0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.84:43406/0 (socket says 172.21.15.84:43406) 2022-12-01T09:47:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.746+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:47:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.746+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.746+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.746+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-12-01T09:47:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.746+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:18.757 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.746+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:18.758 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.746+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:18.758 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.746+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:18.951 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:18.952 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:18.952 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:47:18.952 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:18.953 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:18.953 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-12-01T09:47:18.953 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-12-01T09:47:18.953 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:18.954 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:18.954 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:18.947+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-12-01T09:47:19.352 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:19.352 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:19.353 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-12-01T09:47:19.353 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:19.353 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:19.353 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:47:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-12-01T09:47:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:19.348+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-12-01T09:47:20.153 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:20.154 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:20.154 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:47:20.154 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:20.155 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:20.155 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-12-01T09:47:20.155 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-12-01T09:47:20.155 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:20.156 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:20.156 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:20.149+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-12-01T09:47:21.750 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73d7fa700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 msgr2=0x7fd7400cb890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:21.750 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73d7fa700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd74015b810 0x7fd7400cb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:21.751 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73d7fa700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 msgr2=0x7fd74015f480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:21.751 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73d7fa700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.191:3300/0 conn(0x7fd740070430 0x7fd74015f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:21.751 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73d7fa700 1 -- 172.21.15.84:0/899444440 >> v1:172.21.15.191:6789/0 conn(0x7fd74006ffc0 legacy=0x7fd74006add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:21.751 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73d7fa700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:21.752 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73d7fa700 1 -- 172.21.15.84:0/899444440 --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7400cd920 con 0x7fd724003770 2022-12-01T09:47:21.752 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73d7fa700 1 -- 172.21.15.84:0/899444440 --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7400cd640 con 0x7fd7240073d0 2022-12-01T09:47:21.752 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:21.752 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:47:21.753 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:21.753 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:21.753 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.746+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.947+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.947+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:47:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.947+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:21.951 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.947+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:21.952 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:21.947+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-12-01T09:47:22.352 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:22.348+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:22.352 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:22.348+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:47:22.352 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:22.348+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:22.352 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:22.348+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:22.353 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:22.348+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-12-01T09:47:23.152 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.149+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:23.153 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.149+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:47:23.154 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.149+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:23.154 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.149+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:23.155 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.149+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-12-01T09:47:23.940 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.933+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212492852 0 0) 0x7fd7400cd920 con 0x7fd724003770 2022-12-01T09:47:23.940 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.933+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7200035b0 con 0x7fd724003770 2022-12-01T09:47:23.940 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1394174643 0 0) 0x7fd7200035b0 con 0x7fd724003770 2022-12-01T09:47:23.941 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 msgr2=0x7fd724009880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:23.941 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 --2- 172.21.15.84:0/899444440 >> v2:172.21.15.84:3300/0 conn(0x7fd7240073d0 0x7fd724009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:23.941 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72400af80 con 0x7fd724003770 2022-12-01T09:47:23.941 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2125368202 0 0) 0x7fd72800e590 con 0x7fd724003770 2022-12-01T09:47:23.942 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 >> v1:172.21.15.84:6789/0 conn(0x7fd724003770 legacy=0x7fd724005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:23.942 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73ffff700 1 -- 172.21.15.84:0/899444440 reap_dead start 2022-12-01T09:47:23.942 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd720003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:23.943 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd720004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:23.943 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd72000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:23.943 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400cd640 con 0x7fd720004850 2022-12-01T09:47:23.943 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72000c2a0 con 0x7fd720005400 2022-12-01T09:47:23.944 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72000c460 con 0x7fd720003a50 2022-12-01T09:47:23.944 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd720003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:23.944 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd720004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:23.944 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.934+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 msgr2=0x7fd720003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:47:23.945 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73f7fe700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 msgr2=0x7fd720003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:23.945 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd720003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:23.945 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd720003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:23.945 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73ffff700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd72000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:23.945 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 msgr2=0x7fd720003e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:23.946 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd720003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:23.946 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 msgr2=0x7fd72000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:23.947 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd72000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:23.947 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73effd700 1 -- 172.21.15.84:0/899444440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7200035b0 con 0x7fd720004850 2022-12-01T09:47:23.947 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73effd700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd720004cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd730005ee0 tx=0x7fd720005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fbd34925eab75b server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:23.947 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730017460 con 0x7fd720004850 2022-12-01T09:47:23.947 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7300175c0 con 0x7fd720004850 2022-12-01T09:47:23.948 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.935+0000 7fd73e7fc700 1 -- 172.21.15.84:0/899444440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730017970 con 0x7fd720004850 2022-12-01T09:47:23.948 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 -- 172.21.15.84:0/899444440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 msgr2=0x7fd720004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:23.948 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd720004cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd730005ee0 tx=0x7fd720005880 comp rx=0 tx=0).stop 2022-12-01T09:47:23.948 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 -- 172.21.15.84:0/899444440 shutdown_connections 2022-12-01T09:47:23.949 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd72000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:23.949 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd720004cc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:23.949 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 --2- 172.21.15.84:0/899444440 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd720003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:23.949 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 -- 172.21.15.84:0/899444440 >> 172.21.15.84:0/899444440 conn(0x7fd7400af850 msgr2=0x7fd7400afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:23.949 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 -- 172.21.15.84:0/899444440 shutdown_connections 2022-12-01T09:47:23.950 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.936+0000 7fd745d6b700 1 -- 172.21.15.84:0/899444440 wait complete. 2022-12-01T09:47:23.953 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.938+0000 7fd745d6b700 1 Processor -- start 2022-12-01T09:47:23.953 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.939+0000 7fd745d6b700 1 -- start start 2022-12-01T09:47:23.954 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd745d6b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd7401f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:23.954 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd745d6b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd7401e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:23.954 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd745d6b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd7400c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:23.954 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd745d6b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7401f5670 con 0x7fd720004850 2022-12-01T09:47:23.954 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd745d6b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7401f18d0 con 0x7fd720005400 2022-12-01T09:47:23.955 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd745d6b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7401f1bb0 con 0x7fd720003a50 2022-12-01T09:47:23.955 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd7401e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:23.955 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73f7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd7401f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:23.955 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73f7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd7401f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.84:45104/0 (socket says 172.21.15.84:45104) 2022-12-01T09:47:23.956 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73f7fe700 1 -- 172.21.15.84:0/661279307 learned_addr learned my addr 172.21.15.84:0/661279307 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:23.956 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73ffff700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd7400c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:23.956 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73f7fe700 1 -- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 msgr2=0x7fd7401f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-12-01T09:47:23.956 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73f7fe700 1 -- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 msgr2=0x7fd7401f5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:23.957 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd7401f5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:23.957 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.940+0000 7fd73f7fe700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd7401f5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:23.957 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd73effd700 1 -- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 msgr2=0x7fd7401f5130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:23.957 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd73effd700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd7401f5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:23.957 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd73effd700 1 -- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 msgr2=0x7fd7400c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:23.958 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd73effd700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd7400c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:23.958 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd73effd700 1 -- 172.21.15.84:0/661279307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7401f1e90 con 0x7fd720004850 2022-12-01T09:47:23.958 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd73ffff700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd7400c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:47:23.958 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd73effd700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd7401e4e80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd73001c000 tx=0x7fd730001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:23.959 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd737fff700 1 -- 172.21.15.84:0/661279307 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730027070 con 0x7fd720004850 2022-12-01T09:47:23.959 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd737fff700 1 -- 172.21.15.84:0/661279307 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730007d20 con 0x7fd720004850 2022-12-01T09:47:23.959 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd737fff700 1 -- 172.21.15.84:0/661279307 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730017460 con 0x7fd720004850 2022-12-01T09:47:23.959 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7400c2db0 con 0x7fd720004850 2022-12-01T09:47:23.960 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.941+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7401e55a0 con 0x7fd720004850 2022-12-01T09:47:23.960 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.942+0000 7fd737fff700 1 -- 172.21.15.84:0/661279307 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd730032020 con 0x7fd720004850 2022-12-01T09:47:23.960 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.942+0000 7fd737fff700 1 -- 172.21.15.84:0/661279307 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd730035480 con 0x7fd720004850 2022-12-01T09:47:23.960 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.942+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7401f20b0 con 0x7fd720004850 2022-12-01T09:47:23.960 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:23.945+0000 7fd737fff700 1 -- 172.21.15.84:0/661279307 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fd7401e55a0 con 0x7fd720004850 2022-12-01T09:47:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.022+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd740067420 con 0x7fd720004850 2022-12-01T09:47:24.033 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.029+0000 7fd737fff700 1 -- 172.21.15.84:0/661279307 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd73001b020 con 0x7fd720004850 2022-12-01T09:47:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.030+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 msgr2=0x7fd7401e4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.030+0000 7fd745d6b700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd7401e4e80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd73001c000 tx=0x7fd730001120 comp rx=0 tx=0).stop 2022-12-01T09:47:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.030+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 shutdown_connections 2022-12-01T09:47:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.030+0000 7fd745d6b700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd720005400 0x7fd7400c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.037 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.030+0000 7fd745d6b700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd720003a50 0x7fd7401f5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.037 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.030+0000 7fd745d6b700 1 --2- 172.21.15.84:0/661279307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd720004850 0x7fd7401e4e80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.037 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.030+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 >> 172.21.15.84:0/661279307 conn(0x7fd7400af850 msgr2=0x7fd7400afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:24.037 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.031+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 shutdown_connections 2022-12-01T09:47:24.038 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:24.031+0000 7fd745d6b700 1 -- 172.21.15.84:0/661279307 wait complete. 2022-12-01T09:47:24.038 INFO:teuthology.orchestra.run.smithi084.stderr:adjusted tunables profile to default 2022-12-01T09:47:24.047 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-12-01T09:47:24.047 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:24.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-12-01T09:47:24.060 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:24.057+0000 7f86607ad200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-12-01T09:47:24.064 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:24.061+0000 7f0b7d114200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-12-01T09:47:24.075 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:24.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-12-01T09:47:24.124 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:24.121+0000 7f86607ad200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-12-01T09:47:24.132 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:24.129+0000 7f0b7d114200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-12-01T09:47:24.141 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:24.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-12-01T09:47:24.198 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:24.195+0000 7f86607ad200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-12-01T09:47:24.208 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2022-12-01T09:47:24.209 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-12-01T09:47:24.210 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:24.207+0000 7f0b7d114200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-12-01T09:47:24.237 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:24.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-12-01T09:47:24.265 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:24.265 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-12-01T09:47:24.277 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:24.274+0000 7f86607ad200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-12-01T09:47:24.293 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:24.290+0000 7f0b7d114200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-12-01T09:47:24.303 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:24.303 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-12-01T09:47:24.368 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2022-12-01T09:47:24.369 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-12-01T09:47:24.425 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:24.422+0000 7f86607ad200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-12-01T09:47:24.432 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:24.429+0000 7f0b7d114200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-12-01T09:47:24.435 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new e9c852ba-df6e-4c09-98f8-692b089e63c0 0 2022-12-01T09:47:24.596 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.592+0000 7fcd84227700 1 Processor -- start 2022-12-01T09:47:24.596 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.593+0000 7fcd84227700 1 -- start start 2022-12-01T09:47:24.600 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.593+0000 7fcd84227700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c0c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.600 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.593+0000 7fcd84227700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd7c0cc940 0x7fcd7c071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.600 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.593+0000 7fcd84227700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd7c06e680 0x7fcd7c0724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.601 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.593+0000 7fcd84227700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c0ae810 con 0x7fcd7c0cc940 2022-12-01T09:47:24.601 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.593+0000 7fcd84227700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c0ce2a0 con 0x7fcd7c06e680 2022-12-01T09:47:24.602 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.593+0000 7fcd84227700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0c5740 con 0x7fcd7c15b9d0 2022-12-01T09:47:24.602 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd81fc3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c0c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.602 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd827c4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd7c06e680 0x7fcd7c0724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.602 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd81fc3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c0c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37204/0 (socket says 172.21.15.191:37204) 2022-12-01T09:47:24.603 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd827c4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd7c06e680 0x7fcd7c0724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:47414/0 (socket says 172.21.15.191:47414) 2022-12-01T09:47:24.603 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd81fc3700 1 -- 172.21.15.191:0/3577643203 learned_addr learned my addr 172.21.15.191:0/3577643203 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:24.603 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd817c2700 1 --2- 172.21.15.191:0/3577643203 >> v2:172.21.15.84:3300/0 conn(0x7fcd7c0cc940 0x7fcd7c071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.603 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd81fc3700 1 -- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 msgr2=0x7fcd7c0c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-12-01T09:47:24.604 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd81fc3700 1 -- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 msgr2=0x7fcd7c0c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:24.604 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd81fc3700 1 --2- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c0c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:24.604 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd81fc3700 1 --2- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c0c4b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:24.604 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd827c4700 1 -- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 msgr2=0x7fcd7c0c4b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:24.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd827c4700 1 --2- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c0c4b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd827c4700 1 -- 172.21.15.191:0/3577643203 >> v2:172.21.15.84:3300/0 conn(0x7fcd7c0cc940 msgr2=0x7fcd7c071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:24.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd827c4700 1 --2- 172.21.15.191:0/3577643203 >> v2:172.21.15.84:3300/0 conn(0x7fcd7c0cc940 0x7fcd7c071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd827c4700 1 -- 172.21.15.191:0/3577643203 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c0c5c70 con 0x7fcd7c06e680 2022-12-01T09:47:24.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd827c4700 1 --2- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c06e680 0x7fcd7c0724a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0ae510 tx=0x7fcd7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59dd47153557ef7e server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:24.606 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd80fc1700 1 -- 172.21.15.191:0/3577643203 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400e070 con 0x7fcd7c06e680 2022-12-01T09:47:24.606 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd80fc1700 1 -- 172.21.15.191:0/3577643203 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7400bb20 con 0x7fcd7c06e680 2022-12-01T09:47:24.606 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.594+0000 7fcd80fc1700 1 -- 172.21.15.191:0/3577643203 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74012430 con 0x7fcd7c06e680 2022-12-01T09:47:24.606 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 -- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c06e680 msgr2=0x7fcd7c0724a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.607 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 --2- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c06e680 0x7fcd7c0724a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0ae510 tx=0x7fcd7400b330 comp rx=0 tx=0).stop 2022-12-01T09:47:24.607 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 -- 172.21.15.191:0/3577643203 shutdown_connections 2022-12-01T09:47:24.607 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 --2- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c06e680 0x7fcd7c0724a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.608 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 --2- 172.21.15.191:0/3577643203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c0c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.608 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 --2- 172.21.15.191:0/3577643203 >> v2:172.21.15.84:3300/0 conn(0x7fcd7c0cc940 0x7fcd7c071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.608 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 -- 172.21.15.191:0/3577643203 >> 172.21.15.191:0/3577643203 conn(0x7fcd7c0cb7f0 msgr2=0x7fcd7c0cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:24.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 -- 172.21.15.191:0/3577643203 shutdown_connections 2022-12-01T09:47:24.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.595+0000 7fcd84227700 1 -- 172.21.15.191:0/3577643203 wait complete. 2022-12-01T09:47:24.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.597+0000 7fcd84227700 1 Processor -- start 2022-12-01T09:47:24.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.597+0000 7fcd84227700 1 -- start start 2022-12-01T09:47:24.613 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd84227700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd7c06e680 0x7fcd7c159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.614 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd84227700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c0cc940 0x7fcd7c157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.614 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd84227700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.614 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd84227700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1580c0 con 0x7fcd7c06e680 2022-12-01T09:47:24.614 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd84227700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1583a0 con 0x7fcd7c0cc940 2022-12-01T09:47:24.614 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd84227700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c158680 con 0x7fcd7c15b9d0 2022-12-01T09:47:24.615 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd817c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c0cc940 0x7fcd7c157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.615 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd827c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.615 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd817c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c0cc940 0x7fcd7c157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:47418/0 (socket says 172.21.15.191:47418) 2022-12-01T09:47:24.615 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd817c2700 1 -- 172.21.15.191:0/3844053982 learned_addr learned my addr 172.21.15.191:0/3844053982 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:24.616 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd81fc3700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd7c06e680 0x7fcd7c159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.616 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd827c4700 1 -- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 msgr2=0x7fcd7c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2022-12-01T09:47:24.617 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd827c4700 1 -- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 msgr2=0x7fcd7c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:24.617 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd827c4700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:24.617 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd827c4700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:24.618 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd817c2700 1 -- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 msgr2=0x7fcd7c06caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:24.618 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd817c2700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c06caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.618 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd817c2700 1 -- 172.21.15.191:0/3844053982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd7c06e680 msgr2=0x7fcd7c159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.618 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd817c2700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd7c06e680 0x7fcd7c159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.619 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.598+0000 7fcd817c2700 1 -- 172.21.15.191:0/3844053982 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74009a70 con 0x7fcd7c0cc940 2022-12-01T09:47:24.619 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.599+0000 7fcd817c2700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c0cc940 0x7fcd7c157670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e9b0 tx=0x7fcd6c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:24.619 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.599+0000 7fcd6a7fc700 1 -- 172.21.15.191:0/3844053982 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c009710 con 0x7fcd7c0cc940 2022-12-01T09:47:24.619 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.599+0000 7fcd6a7fc700 1 -- 172.21.15.191:0/3844053982 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c00c850 con 0x7fcd7c0cc940 2022-12-01T09:47:24.620 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.599+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c158960 con 0x7fcd7c0cc940 2022-12-01T09:47:24.620 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.599+0000 7fcd6a7fc700 1 -- 172.21.15.191:0/3844053982 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00cbc0 con 0x7fcd7c0cc940 2022-12-01T09:47:24.620 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.599+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c144ab0 con 0x7fcd7c0cc940 2022-12-01T09:47:24.620 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.599+0000 7fcd6a7fc700 1 -- 172.21.15.191:0/3844053982 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fcd6c019030 con 0x7fcd7c0cc940 2022-12-01T09:47:24.620 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.599+0000 7fcd6a7fc700 1 -- 172.21.15.191:0/3844053982 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcd6c0065c0 con 0x7fcd7c0cc940 2022-12-01T09:47:24.621 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.600+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c158e80 con 0x7fcd7c0cc940 2022-12-01T09:47:24.621 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.602+0000 7fcd6a7fc700 1 -- 172.21.15.191:0/3844053982 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fcd7c158e80 con 0x7fcd7c0cc940 2022-12-01T09:47:24.688 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.685+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e9c852ba-df6e-4c09-98f8-692b089e63c0", "id": 0} v 0) v1 -- 0x7fcd7c0cde80 con 0x7fcd7c0cc940 2022-12-01T09:47:24.694 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.691+0000 7fcd6a7fc700 1 -- 172.21.15.191:0/3844053982 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e9c852ba-df6e-4c09-98f8-692b089e63c0", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcd6c021060 con 0x7fcd7c0cc940 2022-12-01T09:47:24.694 INFO:teuthology.orchestra.run.smithi191.stdout:0 2022-12-01T09:47:24.696 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c0cc940 msgr2=0x7fcd7c157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.696 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c0cc940 0x7fcd7c157670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e9b0 tx=0x7fcd6c00ecc0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.696 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 shutdown_connections 2022-12-01T09:47:24.696 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd7c15b9d0 0x7fcd7c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.697 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd7c06e680 0x7fcd7c159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.697 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 --2- 172.21.15.191:0/3844053982 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7c0cc940 0x7fcd7c157670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.697 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 >> 172.21.15.191:0/3844053982 conn(0x7fcd7c0cb7f0 msgr2=0x7fcd7c0c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:24.697 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 shutdown_connections 2022-12-01T09:47:24.698 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.692+0000 7fcd84227700 1 -- 172.21.15.191:0/3844053982 wait complete. 2022-12-01T09:47:24.707 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 2b47da52-0666-49d8-b263-725119e42db4 1 2022-12-01T09:47:24.765 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:24.761+0000 7f0b7d114200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-12-01T09:47:24.765 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:24.763+0000 7f86607ad200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-12-01T09:47:24.827 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.823+0000 7f9bcc69c700 1 Processor -- start 2022-12-01T09:47:24.827 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.823+0000 7f9bcc69c700 1 -- start start 2022-12-01T09:47:24.829 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.824+0000 7f9bcc69c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9bc40cea50 0x7f9bc41629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.824+0000 7f9bcc69c700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc406b6a0 con 0x7f9bc4157fc0 2022-12-01T09:47:24.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.824+0000 7f9bcc69c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc40cc500 con 0x7f9bc40cc090 2022-12-01T09:47:24.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.824+0000 7f9bcc69c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc406b090 con 0x7f9bc40cea50 2022-12-01T09:47:24.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.824+0000 7f9bcac39700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9bc40cea50 0x7f9bc41629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.824+0000 7f9bc9c37700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f9bc40cc090 0x7f9bc406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:53362/0 (socket says 172.21.15.191:53362) 2022-12-01T09:47:24.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.824+0000 7f9bc9c37700 1 -- 172.21.15.191:0/3657534769 learned_addr learned my addr 172.21.15.191:0/3657534769 (peer_addr_for_me v1:172.21.15.191:0/0) 2022-12-01T09:47:24.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.825+0000 7f9bcac39700 1 -- 172.21.15.191:0/3657534769 >> v1:172.21.15.191:6789/0 conn(0x7f9bc40cc090 legacy=0x7f9bc406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.825+0000 7f9bcac39700 1 -- 172.21.15.191:0/3657534769 >> v1:172.21.15.84:6789/0 conn(0x7f9bc4157fc0 legacy=0x7f9bc406bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.825+0000 7f9bc9436700 1 -- 172.21.15.191:0/3657534769 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936425368 0 0) 0x7f9bc40cc500 con 0x7f9bc40cc090 2022-12-01T09:47:24.832 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.825+0000 7f9bcac39700 1 -- 172.21.15.191:0/3657534769 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40715e0 con 0x7f9bc40cea50 2022-12-01T09:47:24.832 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.825+0000 7f9bcac39700 1 --2- 172.21.15.191:0/3657534769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cea50 0x7f9bc41629e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009d90 tx=0x7f9bbc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71227df55a6ec5d server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:24.832 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.825+0000 7f9bc9436700 1 -- 172.21.15.191:0/3657534769 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00e070 con 0x7f9bc40cea50 2022-12-01T09:47:24.832 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.825+0000 7f9bc9436700 1 -- 172.21.15.191:0/3657534769 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc00bbd0 con 0x7f9bc40cea50 2022-12-01T09:47:24.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.825+0000 7f9bc9436700 1 -- 172.21.15.191:0/3657534769 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc012430 con 0x7f9bc40cea50 2022-12-01T09:47:24.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.826+0000 7f9bcc69c700 1 -- 172.21.15.191:0/3657534769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cea50 msgr2=0x7f9bc41629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.826+0000 7f9bcc69c700 1 --2- 172.21.15.191:0/3657534769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cea50 0x7f9bc41629e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009d90 tx=0x7f9bbc00b330 comp rx=0 tx=0).stop 2022-12-01T09:47:24.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.826+0000 7f9bcc69c700 1 -- 172.21.15.191:0/3657534769 shutdown_connections 2022-12-01T09:47:24.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.826+0000 7f9bcc69c700 1 --2- 172.21.15.191:0/3657534769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cea50 0x7f9bc41629e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.834 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.826+0000 7f9bcc69c700 1 -- 172.21.15.191:0/3657534769 >> 172.21.15.191:0/3657534769 conn(0x7f9bc4070a30 msgr2=0x7f9bc4070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:24.834 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.826+0000 7f9bcc69c700 1 -- 172.21.15.191:0/3657534769 shutdown_connections 2022-12-01T09:47:24.834 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.826+0000 7f9bcc69c700 1 -- 172.21.15.191:0/3657534769 wait complete. 2022-12-01T09:47:24.837 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.827+0000 7f9bcc69c700 1 Processor -- start 2022-12-01T09:47:24.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.827+0000 7f9bcc69c700 1 -- start start 2022-12-01T09:47:24.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bcc69c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cc090 0x7f9bc40c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bcc69c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 0x7f9bc4156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bcc69c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc4157fc0 0x7f9bc40981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:24.839 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bcc69c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40c4800 con 0x7f9bc4157fc0 2022-12-01T09:47:24.839 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bcc69c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc41563f0 con 0x7f9bc40cc090 2022-12-01T09:47:24.839 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bcc69c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc41557e0 con 0x7f9bc40cea50 2022-12-01T09:47:24.839 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bc9c37700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 0x7f9bc4156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.840 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bc9c37700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 0x7f9bc4156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37220/0 (socket says 172.21.15.191:37220) 2022-12-01T09:47:24.840 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bc9c37700 1 -- 172.21.15.191:0/506165323 learned_addr learned my addr 172.21.15.191:0/506165323 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:24.840 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bca438700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cc090 0x7f9bc40c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.840 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bc9c37700 1 -- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 msgr2=0x7f9bc4156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2022-12-01T09:47:24.840 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bc9c37700 1 -- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 msgr2=0x7f9bc4156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:24.841 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bc9c37700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 0x7f9bc4156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:24.841 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bcac39700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc4157fc0 0x7f9bc40981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:24.841 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bc9c37700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 0x7f9bc4156880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:24.841 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bca438700 1 -- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 msgr2=0x7f9bc4156880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:24.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bca438700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 0x7f9bc4156880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bca438700 1 -- 172.21.15.191:0/506165323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc4157fc0 msgr2=0x7f9bc40981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bca438700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc4157fc0 0x7f9bc40981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bca438700 1 -- 172.21.15.191:0/506165323 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc009a70 con 0x7f9bc40cc090 2022-12-01T09:47:24.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bca438700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cc090 0x7f9bc40c42c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c420 tx=0x7f9bc000c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:24.843 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.828+0000 7f9bb2ffd700 1 -- 172.21.15.191:0/506165323 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00052d0 con 0x7f9bc40cc090 2022-12-01T09:47:24.843 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.829+0000 7f9bb2ffd700 1 -- 172.21.15.191:0/506165323 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0005450 con 0x7f9bc40cc090 2022-12-01T09:47:24.843 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.829+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc4155ac0 con 0x7f9bc40cc090 2022-12-01T09:47:24.843 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.829+0000 7f9bb2ffd700 1 -- 172.21.15.191:0/506165323 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0003910 con 0x7f9bc40cc090 2022-12-01T09:47:24.844 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.829+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc4156080 con 0x7f9bc40cc090 2022-12-01T09:47:24.844 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.829+0000 7f9bb2ffd700 1 -- 172.21.15.191:0/506165323 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9bc0019030 con 0x7f9bc40cc090 2022-12-01T09:47:24.844 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.829+0000 7f9bb2ffd700 1 -- 172.21.15.191:0/506165323 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f9bc00118a0 con 0x7f9bc40cc090 2022-12-01T09:47:24.844 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.829+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc41566d0 con 0x7f9bc40cc090 2022-12-01T09:47:24.844 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.831+0000 7f9bb2ffd700 1 -- 172.21.15.191:0/506165323 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f9bc0017070 con 0x7f9bc40cc090 2022-12-01T09:47:24.845 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:24.835+0000 7f0b7d114200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-12-01T09:47:24.846 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:24.840+0000 7f86607ad200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-12-01T09:47:24.919 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.916+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2b47da52-0666-49d8-b263-725119e42db4", "id": 1} v 0) v1 -- 0x7f9bc4067420 con 0x7f9bc40cc090 2022-12-01T09:47:24.942 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.939+0000 7f9bb2ffd700 1 -- 172.21.15.191:0/506165323 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b47da52-0666-49d8-b263-725119e42db4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9bc0003a70 con 0x7f9bc40cc090 2022-12-01T09:47:24.943 INFO:teuthology.orchestra.run.smithi191.stdout:1 2022-12-01T09:47:24.944 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cc090 msgr2=0x7f9bc40c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:24.944 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cc090 0x7f9bc40c42c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000c420 tx=0x7f9bc000c730 comp rx=0 tx=0).stop 2022-12-01T09:47:24.944 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 shutdown_connections 2022-12-01T09:47:24.945 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9bc4157fc0 0x7f9bc40981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.945 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc40cc090 0x7f9bc40c42c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.945 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 --2- 172.21.15.191:0/506165323 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bc40cea50 0x7f9bc4156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:24.945 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 >> 172.21.15.191:0/506165323 conn(0x7f9bc4070a30 msgr2=0x7f9bc40b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:24.945 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 shutdown_connections 2022-12-01T09:47:24.946 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:24.940+0000 7f9bcc69c700 1 -- 172.21.15.191:0/506165323 wait complete. 2022-12-01T09:47:24.955 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 7b668a53-55d9-481c-83a5-df590cd91fad 2 2022-12-01T09:47:24.973 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:24.970+0000 7f0b7d114200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-12-01T09:47:24.983 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:24.979+0000 7f86607ad200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-12-01T09:47:25.053 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:25.049+0000 7f0b7d114200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-12-01T09:47:25.067 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:25.063+0000 7f86607ad200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-12-01T09:47:25.076 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.073+0000 7f13728b1700 1 Processor -- start 2022-12-01T09:47:25.077 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.073+0000 7f13728b1700 1 -- start start 2022-12-01T09:47:25.084 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f13728b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.084 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f13728b1700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c15f7f0 con 0x7f136c15b9d0 2022-12-01T09:47:25.085 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f13728b1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c0cceb0 con 0x7f136c0ae3f0 2022-12-01T09:47:25.085 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f13728b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c15bda0 con 0x7f136c0cc940 2022-12-01T09:47:25.085 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.085 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37226/0 (socket says 172.21.15.191:37226) 2022-12-01T09:47:25.086 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136b7fe700 1 -- 172.21.15.191:0/236397453 learned_addr learned my addr 172.21.15.191:0/236397453 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:25.086 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f1370e4e700 1 --1- 172.21.15.191:0/236397453 >> v1:172.21.15.191:6789/0 conn(0x7f136c0ae3f0 0x7f136c069390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:53376/0 (socket says 172.21.15.191:53376) 2022-12-01T09:47:25.086 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136b7fe700 1 -- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 msgr2=0x7f136c071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:47:25.086 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136b7fe700 1 -- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 msgr2=0x7f136c071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:25.087 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136b7fe700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:25.087 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136b7fe700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c071f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:25.087 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964113787 0 0) 0x7f136c0cceb0 con 0x7f136c0ae3f0 2022-12-01T09:47:25.087 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13580035b0 con 0x7f136c0ae3f0 2022-12-01T09:47:25.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.074+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578637400 0 0) 0x7f136c15f7f0 con 0x7f136c15b9d0 2022-12-01T09:47:25.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f136c0cceb0 con 0x7f136c15b9d0 2022-12-01T09:47:25.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1999048832 0 0) 0x7f13580035b0 con 0x7f136c0ae3f0 2022-12-01T09:47:25.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 msgr2=0x7f136c071f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:25.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c071f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 >> v1:172.21.15.84:6789/0 conn(0x7f136c15b9d0 legacy=0x7f136c15fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0ce2f0 con 0x7f136c0ae3f0 2022-12-01T09:47:25.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2125368202 0 0) 0x7f1364001a30 con 0x7f136c0ae3f0 2022-12-01T09:47:25.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 >> v1:172.21.15.191:6789/0 conn(0x7f136c0ae3f0 legacy=0x7f136c069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f1358003b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13580040d0 0x7f1358004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1358004d90 0x7f1358005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c15bda0 con 0x7f1358004d90 2022-12-01T09:47:25.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135800bac0 con 0x7f13580040d0 2022-12-01T09:47:25.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135800bc50 con 0x7f136c0cc940 2022-12-01T09:47:25.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136bfff700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f1358003b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136b7fe700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13580040d0 0x7f1358004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f1370e4e700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1358004d90 0x7f1358005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136bfff700 1 -- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13580040d0 msgr2=0x7f1358004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:25.092 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136bfff700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13580040d0 0x7f1358004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.092 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136bfff700 1 -- 172.21.15.191:0/236397453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1358004d90 msgr2=0x7f1358005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:25.092 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136bfff700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1358004d90 0x7f1358005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.092 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136bfff700 1 -- 172.21.15.191:0/236397453 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13580035b0 con 0x7f136c0cc940 2022-12-01T09:47:25.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136b7fe700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13580040d0 0x7f1358004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:47:25.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136bfff700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f1358003b90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f135c001940 tx=0x7f1358005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329b9f3dc3797922 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:25.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.075+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c001430 con 0x7f136c0cc940 2022-12-01T09:47:25.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c001d70 con 0x7f136c0cc940 2022-12-01T09:47:25.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f136affd700 1 -- 172.21.15.191:0/236397453 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c005510 con 0x7f136c0cc940 2022-12-01T09:47:25.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f13728b1700 1 -- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 msgr2=0x7f1358003b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f13728b1700 1 --2- 172.21.15.191:0/236397453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f1358003b90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f135c001940 tx=0x7f1358005880 comp rx=0 tx=0).stop 2022-12-01T09:47:25.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f1370e4e700 1 -- 172.21.15.191:0/236397453 reap_dead start 2022-12-01T09:47:25.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f13728b1700 1 -- 172.21.15.191:0/236397453 shutdown_connections 2022-12-01T09:47:25.095 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f13728b1700 1 -- 172.21.15.191:0/236397453 >> 172.21.15.191:0/236397453 conn(0x7f136c0cb7f0 msgr2=0x7f136c0cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:25.095 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f13728b1700 1 -- 172.21.15.191:0/236397453 shutdown_connections 2022-12-01T09:47:25.095 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.076+0000 7f13728b1700 1 -- 172.21.15.191:0/236397453 wait complete. 2022-12-01T09:47:25.095 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.078+0000 7f13728b1700 1 Processor -- start 2022-12-01T09:47:25.096 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.078+0000 7f13728b1700 1 -- start start 2022-12-01T09:47:25.096 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f13728b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c14fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.096 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f13728b1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c0b1ce0 0x7f136c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.096 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f13728b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0b3b50 0x7f136c0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.096 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f13728b1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c160390 con 0x7f136c0b1ce0 2022-12-01T09:47:25.097 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f13728b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0b2690 con 0x7f136c0b3b50 2022-12-01T09:47:25.097 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f13728b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c06e1c0 con 0x7f136c0cc940 2022-12-01T09:47:25.097 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f136bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c14fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.097 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f1370e4e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0b3b50 0x7f136c0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.098 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f136bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c14fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37236/0 (socket says 172.21.15.191:37236) 2022-12-01T09:47:25.098 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f136bfff700 1 -- 172.21.15.191:0/3902581313 learned_addr learned my addr 172.21.15.191:0/3902581313 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:25.098 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f136b7fe700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c0b1ce0 0x7f136c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.098 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f136bfff700 1 -- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 msgr2=0x7f136c14fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-12-01T09:47:25.098 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f136bfff700 1 -- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 msgr2=0x7f136c14fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:25.099 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f136bfff700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c14fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:25.099 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.079+0000 7f136bfff700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c14fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:25.099 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1370e4e700 1 -- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 msgr2=0x7f136c14fa30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:25.099 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1370e4e700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c14fa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.100 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1370e4e700 1 -- 172.21.15.191:0/3902581313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c0b1ce0 msgr2=0x7f136c0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.100 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1370e4e700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c0b1ce0 0x7f136c0b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.100 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1370e4e700 1 -- 172.21.15.191:0/3902581313 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0ab8a0 con 0x7f136c0b3b50 2022-12-01T09:47:25.100 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1370e4e700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0b3b50 0x7f136c0b3ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f136c0717c0 tx=0x7f1364005a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:25.100 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1368ff9700 1 -- 172.21.15.191:0/3902581313 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364008330 con 0x7f136c0b3b50 2022-12-01T09:47:25.101 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1368ff9700 1 -- 172.21.15.191:0/3902581313 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13640084b0 con 0x7f136c0b3b50 2022-12-01T09:47:25.101 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1368ff9700 1 -- 172.21.15.191:0/3902581313 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364008860 con 0x7f136c0b3b50 2022-12-01T09:47:25.101 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c0b4830 con 0x7f136c0b3b50 2022-12-01T09:47:25.101 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c0abb60 con 0x7f136c0b3b50 2022-12-01T09:47:25.102 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1368ff9700 1 -- 172.21.15.191:0/3902581313 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f13640089c0 con 0x7f136c0b3b50 2022-12-01T09:47:25.102 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.080+0000 7f1368ff9700 1 -- 172.21.15.191:0/3902581313 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f136401eb00 con 0x7f136c0b3b50 2022-12-01T09:47:25.102 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.081+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c067420 con 0x7f136c0b3b50 2022-12-01T09:47:25.103 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.085+0000 7f1368ff9700 1 -- 172.21.15.191:0/3902581313 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f1364023060 con 0x7f136c0b3b50 2022-12-01T09:47:25.170 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.167+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7b668a53-55d9-481c-83a5-df590cd91fad", "id": 2} v 0) v1 -- 0x7f136c0ce570 con 0x7f136c0b3b50 2022-12-01T09:47:25.178 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.174+0000 7f1368ff9700 1 -- 172.21.15.191:0/3902581313 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b668a53-55d9-481c-83a5-df590cd91fad", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f136401a070 con 0x7f136c0b3b50 2022-12-01T09:47:25.178 INFO:teuthology.orchestra.run.smithi191.stdout:2 2022-12-01T09:47:25.179 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.175+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0b3b50 msgr2=0x7f136c0b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.175+0000 7f13728b1700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0b3b50 0x7f136c0b3ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f136c0717c0 tx=0x7f1364005a40 comp rx=0 tx=0).stop 2022-12-01T09:47:25.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.176+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 shutdown_connections 2022-12-01T09:47:25.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.176+0000 7f13728b1700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0b3b50 0x7f136c0b3ff0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.176+0000 7f13728b1700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cc940 0x7f136c14fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.181 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.176+0000 7f13728b1700 1 --2- 172.21.15.191:0/3902581313 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f136c0b1ce0 0x7f136c0b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.181 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.176+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 >> 172.21.15.191:0/3902581313 conn(0x7f136c0cb7f0 msgr2=0x7f136c0cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:25.181 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.176+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 shutdown_connections 2022-12-01T09:47:25.181 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.176+0000 7f13728b1700 1 -- 172.21.15.191:0/3902581313 wait complete. 2022-12-01T09:47:25.191 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 1f2c87e3-7398-4f50-a56c-5f798d2fb279 3 2022-12-01T09:47:25.352 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.349+0000 7f7e602fc700 1 Processor -- start 2022-12-01T09:47:25.353 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.349+0000 7f7e602fc700 1 -- start start 2022-12-01T09:47:25.356 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e602fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e5815be80 0x7f7e5806a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.356 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e602fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e58071f60 0x7f7e580723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.356 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e602fc700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e58072900 con 0x7f7e58070f60 2022-12-01T09:47:25.357 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e602fc700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e58070b40 con 0x7f7e5815be80 2022-12-01T09:47:25.357 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e602fc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5815c250 con 0x7f7e58071f60 2022-12-01T09:47:25.357 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e5e899700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e58071f60 0x7f7e580723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.357 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e5e899700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7e58071f60 0x7f7e580723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:47454/0 (socket says 172.21.15.191:47454) 2022-12-01T09:47:25.357 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e5e899700 1 -- 172.21.15.191:0/2223131360 learned_addr learned my addr 172.21.15.191:0/2223131360 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:25.358 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.350+0000 7f7e5e098700 1 --2- 172.21.15.191:0/2223131360 >> v2:172.21.15.84:3300/0 conn(0x7f7e5815be80 0x7f7e5806a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.358 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e5e899700 1 -- 172.21.15.191:0/2223131360 >> v2:172.21.15.84:3300/0 conn(0x7f7e5815be80 msgr2=0x7f7e5806a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:25.358 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e5e899700 1 --2- 172.21.15.191:0/2223131360 >> v2:172.21.15.84:3300/0 conn(0x7f7e5815be80 0x7f7e5806a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.358 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e5e899700 1 -- 172.21.15.191:0/2223131360 >> v1:172.21.15.84:6789/0 conn(0x7f7e58070f60 legacy=0x7f7e580cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.359 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e5e899700 1 -- 172.21.15.191:0/2223131360 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e580ce580 con 0x7f7e58071f60 2022-12-01T09:47:25.359 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e5e899700 1 --2- 172.21.15.191:0/2223131360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58071f60 0x7f7e580723c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e58071b20 tx=0x7f7e48009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4854486c096d8402 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:25.359 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e5d096700 1 -- 172.21.15.191:0/2223131360 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48011830 con 0x7f7e58071f60 2022-12-01T09:47:25.360 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e5d096700 1 -- 172.21.15.191:0/2223131360 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e48011990 con 0x7f7e58071f60 2022-12-01T09:47:25.360 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e5d096700 1 -- 172.21.15.191:0/2223131360 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48011d20 con 0x7f7e58071f60 2022-12-01T09:47:25.360 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e602fc700 1 -- 172.21.15.191:0/2223131360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58071f60 msgr2=0x7f7e580723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.360 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e602fc700 1 --2- 172.21.15.191:0/2223131360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58071f60 0x7f7e580723c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e58071b20 tx=0x7f7e48009bd0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.361 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e602fc700 1 -- 172.21.15.191:0/2223131360 shutdown_connections 2022-12-01T09:47:25.361 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e602fc700 1 --2- 172.21.15.191:0/2223131360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58071f60 0x7f7e580723c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.361 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e602fc700 1 --2- 172.21.15.191:0/2223131360 >> v2:172.21.15.84:3300/0 conn(0x7f7e5815be80 0x7f7e5806a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.361 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.351+0000 7f7e602fc700 1 -- 172.21.15.191:0/2223131360 >> 172.21.15.191:0/2223131360 conn(0x7f7e580cb7e0 msgr2=0x7f7e580cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:25.361 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.352+0000 7f7e602fc700 1 -- 172.21.15.191:0/2223131360 shutdown_connections 2022-12-01T09:47:25.362 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.352+0000 7f7e602fc700 1 -- 172.21.15.191:0/2223131360 wait complete. 2022-12-01T09:47:25.362 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e602fc700 1 Processor -- start 2022-12-01T09:47:25.362 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e602fc700 1 -- start start 2022-12-01T09:47:25.366 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e602fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58070f60 0x7f7e5815e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.366 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e602fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 0x7f7e5815cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.366 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e602fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e5815be80 0x7f7e580ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.366 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e602fc700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580c5e30 con 0x7f7e5815be80 2022-12-01T09:47:25.367 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e602fc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580c69b0 con 0x7f7e58070f60 2022-12-01T09:47:25.367 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e602fc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5815d310 con 0x7f7e58071f60 2022-12-01T09:47:25.368 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.353+0000 7f7e5e098700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58070f60 0x7f7e5815e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.368 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e098700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58070f60 0x7f7e5815e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:47456/0 (socket says 172.21.15.191:47456) 2022-12-01T09:47:25.368 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e098700 1 -- 172.21.15.191:0/618188481 learned_addr learned my addr 172.21.15.191:0/618188481 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:25.369 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5d897700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 0x7f7e5815cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.369 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e899700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e5815be80 0x7f7e580ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.370 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5d897700 1 -- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 msgr2=0x7f7e5815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2022-12-01T09:47:25.370 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5d897700 1 -- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 msgr2=0x7f7e5815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:25.371 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5d897700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 0x7f7e5815cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:25.371 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5d897700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 0x7f7e5815cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:25.371 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e098700 1 -- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 msgr2=0x7f7e5815cdd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:25.372 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e098700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 0x7f7e5815cdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.372 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e098700 1 -- 172.21.15.191:0/618188481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e5815be80 msgr2=0x7f7e580ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.373 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e098700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e5815be80 0x7f7e580ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.373 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e098700 1 -- 172.21.15.191:0/618188481 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48009550 con 0x7f7e58070f60 2022-12-01T09:47:25.374 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e5e098700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58070f60 0x7f7e5815e0d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000efc0 tx=0x7f7e5000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:25.374 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e56ffd700 1 -- 172.21.15.191:0/618188481 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000cdc0 con 0x7f7e58070f60 2022-12-01T09:47:25.374 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e56ffd700 1 -- 172.21.15.191:0/618188481 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000f040 con 0x7f7e58070f60 2022-12-01T09:47:25.375 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e580c6080 con 0x7f7e58070f60 2022-12-01T09:47:25.375 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e56ffd700 1 -- 172.21.15.191:0/618188481 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e500146a0 con 0x7f7e58070f60 2022-12-01T09:47:25.376 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e580c6640 con 0x7f7e58070f60 2022-12-01T09:47:25.376 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e56ffd700 1 -- 172.21.15.191:0/618188481 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f7e50014800 con 0x7f7e58070f60 2022-12-01T09:47:25.377 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.354+0000 7f7e56ffd700 1 -- 172.21.15.191:0/618188481 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f7e5001dc50 con 0x7f7e58070f60 2022-12-01T09:47:25.377 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.355+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e58067420 con 0x7f7e58070f60 2022-12-01T09:47:25.378 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.357+0000 7f7e56ffd700 1 -- 172.21.15.191:0/618188481 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f7e5003b6e0 con 0x7f7e58070f60 2022-12-01T09:47:25.443 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.440+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1f2c87e3-7398-4f50-a56c-5f798d2fb279", "id": 3} v 0) v1 -- 0x7f7e5815e720 con 0x7f7e58070f60 2022-12-01T09:47:25.451 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.448+0000 7f7e56ffd700 1 -- 172.21.15.191:0/618188481 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1f2c87e3-7398-4f50-a56c-5f798d2fb279", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7e5002a070 con 0x7f7e58070f60 2022-12-01T09:47:25.452 INFO:teuthology.orchestra.run.smithi191.stdout:3 2022-12-01T09:47:25.453 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58070f60 msgr2=0x7f7e5815e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.453 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58070f60 0x7f7e5815e0d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000efc0 tx=0x7f7e5000c3a0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.453 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 shutdown_connections 2022-12-01T09:47:25.454 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e5815be80 0x7f7e580ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.454 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e58070f60 0x7f7e5815e0d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.454 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 --2- 172.21.15.191:0/618188481 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7e58071f60 0x7f7e5815cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.454 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 >> 172.21.15.191:0/618188481 conn(0x7f7e580cb7e0 msgr2=0x7f7e5815b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:25.455 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 shutdown_connections 2022-12-01T09:47:25.455 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.449+0000 7f7e602fc700 1 -- 172.21.15.191:0/618188481 wait complete. 2022-12-01T09:47:25.465 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 8a4b43a4-6c63-48ab-826a-cb8919c44b8f 4 2022-12-01T09:47:25.480 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:25.477+0000 7f0b7d114200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-12-01T09:47:25.519 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:25.516+0000 7f86607ad200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-12-01T09:47:25.587 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.584+0000 7f9fc0e01700 1 Processor -- start 2022-12-01T09:47:25.588 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.584+0000 7f9fc0e01700 1 -- start start 2022-12-01T09:47:25.590 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.585+0000 7f9fc0e01700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fbc06e3f0 0x7f9fbc0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.591 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.585+0000 7f9fc0e01700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 0x7f9fbc0723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.591 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.585+0000 7f9fc0e01700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc072900 con 0x7f9fbc159580 2022-12-01T09:47:25.591 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.585+0000 7f9fc0e01700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc0c5c80 con 0x7f9fbc06e3f0 2022-12-01T09:47:25.592 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.585+0000 7f9fc0e01700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0cb640 con 0x7f9fbc071f60 2022-12-01T09:47:25.592 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.585+0000 7f9fbad9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 0x7f9fbc0723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.592 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.585+0000 7f9fbad9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 0x7f9fbc0723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37258/0 (socket says 172.21.15.191:37258) 2022-12-01T09:47:25.593 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.585+0000 7f9fbad9d700 1 -- 172.21.15.191:0/2364281581 learned_addr learned my addr 172.21.15.191:0/2364281581 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:25.593 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb9d9b700 1 --2- 172.21.15.191:0/2364281581 >> v2:172.21.15.84:3300/0 conn(0x7f9fbc06e3f0 0x7f9fbc0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.594 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fbad9d700 1 -- 172.21.15.191:0/2364281581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 msgr2=0x7f9fbc0723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-12-01T09:47:25.594 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fbad9d700 1 -- 172.21.15.191:0/2364281581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 msgr2=0x7f9fbc0723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:25.594 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fbad9d700 1 --2- 172.21.15.191:0/2364281581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 0x7f9fbc0723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:25.594 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fbad9d700 1 --2- 172.21.15.191:0/2364281581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 0x7f9fbc0723c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:25.595 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb9d9b700 1 -- 172.21.15.191:0/2364281581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 msgr2=0x7f9fbc0723c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:25.595 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb9d9b700 1 --2- 172.21.15.191:0/2364281581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 0x7f9fbc0723c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.595 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb9d9b700 1 -- 172.21.15.191:0/2364281581 >> v1:172.21.15.84:6789/0 conn(0x7f9fbc159580 legacy=0x7f9fbc06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.596 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb9d9b700 1 -- 172.21.15.191:0/2364281581 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc06e860 con 0x7f9fbc06e3f0 2022-12-01T09:47:25.596 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb959a700 1 -- 172.21.15.191:0/2364281581 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509755836 0 0) 0x7f9fbc072900 con 0x7f9fbc159580 2022-12-01T09:47:25.597 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb9d9b700 1 --2- 172.21.15.191:0/2364281581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc06e3f0 0x7f9fbc0c5740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009840 tx=0x7f9fb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40e45b719d035e7 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:25.597 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb959a700 1 -- 172.21.15.191:0/2364281581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0011830 con 0x7f9fbc06e3f0 2022-12-01T09:47:25.597 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb959a700 1 -- 172.21.15.191:0/2364281581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb0011990 con 0x7f9fbc06e3f0 2022-12-01T09:47:25.597 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.586+0000 7f9fb959a700 1 -- 172.21.15.191:0/2364281581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0011d20 con 0x7f9fbc06e3f0 2022-12-01T09:47:25.598 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.587+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2364281581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc06e3f0 msgr2=0x7f9fbc0c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.598 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.587+0000 7f9fc0e01700 1 --2- 172.21.15.191:0/2364281581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc06e3f0 0x7f9fbc0c5740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009840 tx=0x7f9fb0009b50 comp rx=0 tx=0).stop 2022-12-01T09:47:25.598 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.587+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2364281581 shutdown_connections 2022-12-01T09:47:25.599 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.587+0000 7f9fc0e01700 1 --2- 172.21.15.191:0/2364281581 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc071f60 0x7f9fbc0723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.599 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.587+0000 7f9fc0e01700 1 --2- 172.21.15.191:0/2364281581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc06e3f0 0x7f9fbc0c5740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.599 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.587+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2364281581 >> 172.21.15.191:0/2364281581 conn(0x7f9fbc0cb7f0 msgr2=0x7f9fbc0cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:25.599 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.587+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2364281581 shutdown_connections 2022-12-01T09:47:25.600 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.587+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2364281581 wait complete. 2022-12-01T09:47:25.603 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.589+0000 7f9fc0e01700 1 Processor -- start 2022-12-01T09:47:25.604 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.589+0000 7f9fc0e01700 1 -- start start 2022-12-01T09:47:25.604 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.589+0000 7f9fc0e01700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 0x7f9fbc1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.604 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.589+0000 7f9fc0e01700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc071f60 0x7f9fbc0c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.589+0000 7f9fc0e01700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc159580 0x7f9fbc0c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.589+0000 7f9fc0e01700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc1503a0 con 0x7f9fbc159580 2022-12-01T09:47:25.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.589+0000 7f9fc0e01700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc150600 con 0x7f9fbc071f60 2022-12-01T09:47:25.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.589+0000 7f9fc0e01700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc1508e0 con 0x7f9fbc06e3f0 2022-12-01T09:47:25.605 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fba59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 0x7f9fbc1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.606 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fb9d9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc071f60 0x7f9fbc0c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.606 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fba59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 0x7f9fbc1513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37260/0 (socket says 172.21.15.191:37260) 2022-12-01T09:47:25.606 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fba59c700 1 -- 172.21.15.191:0/2561546504 learned_addr learned my addr 172.21.15.191:0/2561546504 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:25.606 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fbad9d700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc159580 0x7f9fbc0c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.607 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fba59c700 1 -- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 msgr2=0x7f9fbc1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-12-01T09:47:25.607 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fba59c700 1 -- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 msgr2=0x7f9fbc1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:25.607 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fba59c700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 0x7f9fbc1513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:25.607 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fba59c700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 0x7f9fbc1513c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:25.607 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fb9d9b700 1 -- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 msgr2=0x7f9fbc1513c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:25.608 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fb9d9b700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 0x7f9fbc1513c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.608 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fb9d9b700 1 -- 172.21.15.191:0/2561546504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc159580 msgr2=0x7f9fbc0c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.608 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fb9d9b700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc159580 0x7f9fbc0c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.608 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fb9d9b700 1 -- 172.21.15.191:0/2561546504 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00094a0 con 0x7f9fbc071f60 2022-12-01T09:47:25.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fbad9d700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc159580 0x7f9fbc0c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:47:25.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fb9d9b700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc071f60 0x7f9fbc0c8540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0016000 tx=0x7f9fb0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:25.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9faaffd700 1 -- 172.21.15.191:0/2561546504 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00045d0 con 0x7f9fbc071f60 2022-12-01T09:47:25.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9faaffd700 1 -- 172.21.15.191:0/2561546504 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb001e070 con 0x7f9fbc071f60 2022-12-01T09:47:25.609 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc150bc0 con 0x7f9fbc071f60 2022-12-01T09:47:25.610 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9faaffd700 1 -- 172.21.15.191:0/2561546504 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00038c0 con 0x7f9fbc071f60 2022-12-01T09:47:25.610 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc0b0210 con 0x7f9fbc071f60 2022-12-01T09:47:25.610 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9faaffd700 1 -- 172.21.15.191:0/2561546504 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9fb000f050 con 0x7f9fbc071f60 2022-12-01T09:47:25.611 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.590+0000 7f9faaffd700 1 -- 172.21.15.191:0/2561546504 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f9fb002dd10 con 0x7f9fbc071f60 2022-12-01T09:47:25.611 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.591+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc1510e0 con 0x7f9fbc071f60 2022-12-01T09:47:25.611 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.593+0000 7f9faaffd700 1 -- 172.21.15.191:0/2561546504 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f9fbc1510e0 con 0x7f9fbc071f60 2022-12-01T09:47:25.679 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.676+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8a4b43a4-6c63-48ab-826a-cb8919c44b8f", "id": 4} v 0) v1 -- 0x7f9fbc0c9080 con 0x7f9fbc071f60 2022-12-01T09:47:25.685 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.682+0000 7f9faaffd700 1 -- 172.21.15.191:0/2561546504 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8a4b43a4-6c63-48ab-826a-cb8919c44b8f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9fb0020070 con 0x7f9fbc071f60 2022-12-01T09:47:25.686 INFO:teuthology.orchestra.run.smithi191.stdout:4 2022-12-01T09:47:25.687 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc071f60 msgr2=0x7f9fbc0c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.687 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc071f60 0x7f9fbc0c8540 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0016000 tx=0x7f9fb0004750 comp rx=0 tx=0).stop 2022-12-01T09:47:25.688 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 shutdown_connections 2022-12-01T09:47:25.688 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fbc159580 0x7f9fbc0c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.688 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9fbc06e3f0 0x7f9fbc1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.688 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 --2- 172.21.15.191:0/2561546504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fbc071f60 0x7f9fbc0c8540 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.689 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 >> 172.21.15.191:0/2561546504 conn(0x7f9fbc0cb7f0 msgr2=0x7f9fbc069390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:25.689 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 shutdown_connections 2022-12-01T09:47:25.689 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.683+0000 7f9fc0e01700 1 -- 172.21.15.191:0/2561546504 wait complete. 2022-12-01T09:47:25.698 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 18a3160f-c761-49e7-8d7d-56fdf4c8a7f0 5 2022-12-01T09:47:25.819 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.815+0000 7f3f420b3700 1 Processor -- start 2022-12-01T09:47:25.819 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.815+0000 7f3f420b3700 1 -- start start 2022-12-01T09:47:25.828 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.816+0000 7f3f420b3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.829 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.816+0000 7f3f420b3700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f3c0cb3e0 0x7f3f3c069390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.829 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.816+0000 7f3f420b3700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3c0c9e00 con 0x7f3f3c0cd350 2022-12-01T09:47:25.829 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.816+0000 7f3f420b3700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3c0ca0e0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.816+0000 7f3f420b3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0ca3c0 con 0x7f3f3c0c44f0 2022-12-01T09:47:25.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.816+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.816+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37268/0 (socket says 172.21.15.191:37268) 2022-12-01T09:47:25.830 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.816+0000 7f3f3affd700 1 -- 172.21.15.191:0/855922963 learned_addr learned my addr 172.21.15.191:0/855922963 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:25.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3bfff700 1 --2- 172.21.15.191:0/855922963 >> v2:172.21.15.84:3300/0 conn(0x7f3f3c0cb3e0 0x7f3f3c069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3affd700 1 -- 172.21.15.191:0/855922963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 msgr2=0x7f3f3c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:47:25.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3affd700 1 -- 172.21.15.191:0/855922963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 msgr2=0x7f3f3c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:25.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3affd700 1 --2- 172.21.15.191:0/855922963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:25.831 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3affd700 1 --2- 172.21.15.191:0/855922963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c0caea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:25.832 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3bfff700 1 -- 172.21.15.191:0/855922963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 msgr2=0x7f3f3c0caea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:25.832 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3bfff700 1 --2- 172.21.15.191:0/855922963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c0caea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.832 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3bfff700 1 -- 172.21.15.191:0/855922963 >> v1:172.21.15.84:6789/0 conn(0x7f3f3c0cd350 legacy=0x7f3f3c0cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.832 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3bfff700 1 -- 172.21.15.191:0/855922963 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c0ca6a0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3bfff700 1 --2- 172.21.15.191:0/855922963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cb3e0 0x7f3f3c069390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400bfa0 tx=0x7f3f3400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf86f1768a94d51d server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:25.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3a7fc700 1 -- 172.21.15.191:0/855922963 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34010070 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3a7fc700 1 -- 172.21.15.191:0/855922963 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3400b2e0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.833 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.817+0000 7f3f3a7fc700 1 -- 172.21.15.191:0/855922963 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3400dcf0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.834 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.818+0000 7f3f420b3700 1 -- 172.21.15.191:0/855922963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cb3e0 msgr2=0x7f3f3c069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.834 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.818+0000 7f3f420b3700 1 --2- 172.21.15.191:0/855922963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cb3e0 0x7f3f3c069390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400bfa0 tx=0x7f3f3400d330 comp rx=0 tx=0).stop 2022-12-01T09:47:25.834 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.818+0000 7f3f420b3700 1 -- 172.21.15.191:0/855922963 shutdown_connections 2022-12-01T09:47:25.834 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.818+0000 7f3f420b3700 1 --2- 172.21.15.191:0/855922963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cb3e0 0x7f3f3c069390 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.834 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.818+0000 7f3f420b3700 1 --2- 172.21.15.191:0/855922963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.835 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.818+0000 7f3f420b3700 1 -- 172.21.15.191:0/855922963 >> 172.21.15.191:0/855922963 conn(0x7f3f3c0cb7f0 msgr2=0x7f3f3c0cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:25.835 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.818+0000 7f3f420b3700 1 -- 172.21.15.191:0/855922963 shutdown_connections 2022-12-01T09:47:25.835 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.818+0000 7f3f420b3700 1 -- 172.21.15.191:0/855922963 wait complete. 2022-12-01T09:47:25.836 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.819+0000 7f3f420b3700 1 Processor -- start 2022-12-01T09:47:25.836 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.819+0000 7f3f420b3700 1 -- start start 2022-12-01T09:47:25.836 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f420b3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c14ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.836 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f420b3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f3c0cb3e0 0x7f3f3c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.836 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f420b3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cd350 0x7f3f3c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:25.837 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f420b3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c070780 con 0x7f3f3c0cd350 2022-12-01T09:47:25.837 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f420b3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c14cae0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.837 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f420b3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c14cdc0 con 0x7f3f3c0c44f0 2022-12-01T09:47:25.837 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c14ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f3c0cb3e0 0x7f3f3c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c14ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37272/0 (socket says 172.21.15.191:37272) 2022-12-01T09:47:25.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3b7fe700 1 -- 172.21.15.191:0/3121831114 learned_addr learned my addr 172.21.15.191:0/3121831114 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:25.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3bfff700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cd350 0x7f3f3c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:25.838 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3b7fe700 1 -- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 msgr2=0x7f3f3c14ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-12-01T09:47:25.839 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3b7fe700 1 -- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 msgr2=0x7f3f3c14ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:25.839 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3b7fe700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c14ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:25.839 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.820+0000 7f3f3b7fe700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c14ae10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:25.840 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f3affd700 1 -- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 msgr2=0x7f3f3c14ae10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:25.840 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f3affd700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c14ae10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.840 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f3affd700 1 -- 172.21.15.191:0/3121831114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cd350 msgr2=0x7f3f3c156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.841 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f3affd700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cd350 0x7f3f3c156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.841 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f3affd700 1 -- 172.21.15.191:0/3121831114 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3400bc80 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.841 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f3affd700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f3c0cb3e0 0x7f3f3c1565f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00e7e0 tx=0x7f3f2c00eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:25.841 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f23fff700 1 -- 172.21.15.191:0/3121831114 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c00c070 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f23fff700 1 -- 172.21.15.191:0/3121831114 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c004020 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c14d0a0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f23fff700 1 -- 172.21.15.191:0/3121831114 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0043d0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.842 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c14d660 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.843 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f23fff700 1 -- 172.21.15.191:0/3121831114 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3f2c0036a0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.843 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.821+0000 7f3f23fff700 1 -- 172.21.15.191:0/3121831114 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f3f2c004530 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.843 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.822+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c0c60e0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.843 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.824+0000 7f3f23fff700 1 -- 172.21.15.191:0/3121831114 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f3f3c0c60e0 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.914 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.909+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "18a3160f-c761-49e7-8d7d-56fdf4c8a7f0", "id": 5} v 0) v1 -- 0x7f3f3c067420 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.924 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.921+0000 7f3f23fff700 1 -- 172.21.15.191:0/3121831114 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "18a3160f-c761-49e7-8d7d-56fdf4c8a7f0", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3f2c017030 con 0x7f3f3c0cb3e0 2022-12-01T09:47:25.925 INFO:teuthology.orchestra.run.smithi191.stdout:5 2022-12-01T09:47:25.926 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f3c0cb3e0 msgr2=0x7f3f3c1565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:25.926 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f3c0cb3e0 0x7f3f3c1565f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00e7e0 tx=0x7f3f2c00eaf0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.926 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 shutdown_connections 2022-12-01T09:47:25.927 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cd350 0x7f3f3c156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.927 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f3c0c44f0 0x7f3f3c14ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.927 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 --2- 172.21.15.191:0/3121831114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f3c0cb3e0 0x7f3f3c1565f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:25.927 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 >> 172.21.15.191:0/3121831114 conn(0x7f3f3c0cb7f0 msgr2=0x7f3f3c0cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:25.928 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 shutdown_connections 2022-12-01T09:47:25.928 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:25.922+0000 7f3f420b3700 1 -- 172.21.15.191:0/3121831114 wait complete. 2022-12-01T09:47:25.940 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new e70b8522-3456-479c-81c9-10e06ad3172f 6 2022-12-01T09:47:26.029 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:26.026+0000 7f0b7d114200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-12-01T09:47:26.074 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.071+0000 7fe27ac3d700 1 Processor -- start 2022-12-01T09:47:26.074 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.071+0000 7fe27ac3d700 1 -- start start 2022-12-01T09:47:26.077 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.071+0000 7fe27ac3d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe2740aef30 0x7fe27406d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.078 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.071+0000 7fe27ac3d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe2740b2370 0x7fe274160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.078 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.071+0000 7fe27ac3d700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27415f970 con 0x7fe2740cbc90 2022-12-01T09:47:26.078 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.071+0000 7fe27ac3d700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27406d6c0 con 0x7fe2740aef30 2022-12-01T09:47:26.078 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.071+0000 7fe27ac3d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27406d9a0 con 0x7fe2740b2370 2022-12-01T09:47:26.079 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe273fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe2740b2370 0x7fe274160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.079 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe273fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe2740b2370 0x7fe274160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:47488/0 (socket says 172.21.15.191:47488) 2022-12-01T09:47:26.079 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe273fff700 1 -- 172.21.15.191:0/1086216950 learned_addr learned my addr 172.21.15.191:0/1086216950 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:26.080 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe2789d9700 1 --2- 172.21.15.191:0/1086216950 >> v2:172.21.15.84:3300/0 conn(0x7fe2740aef30 0x7fe27406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.080 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe273fff700 1 -- 172.21.15.191:0/1086216950 >> v2:172.21.15.84:3300/0 conn(0x7fe2740aef30 msgr2=0x7fe27406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:26.080 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe273fff700 1 --2- 172.21.15.191:0/1086216950 >> v2:172.21.15.84:3300/0 conn(0x7fe2740aef30 0x7fe27406d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.080 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe273fff700 1 -- 172.21.15.191:0/1086216950 >> v1:172.21.15.84:6789/0 conn(0x7fe2740cbc90 legacy=0x7fe2740cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.081 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe273fff700 1 -- 172.21.15.191:0/1086216950 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27415fc30 con 0x7fe2740b2370 2022-12-01T09:47:26.081 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe273fff700 1 --2- 172.21.15.191:0/1086216950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740b2370 0x7fe274160150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe27406e540 tx=0x7fe260009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f2272e420d7db3 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:26.081 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe2737fe700 1 -- 172.21.15.191:0/1086216950 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe260011830 con 0x7fe2740b2370 2022-12-01T09:47:26.081 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe2737fe700 1 -- 172.21.15.191:0/1086216950 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe260011990 con 0x7fe2740b2370 2022-12-01T09:47:26.082 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.072+0000 7fe2737fe700 1 -- 172.21.15.191:0/1086216950 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe260011d20 con 0x7fe2740b2370 2022-12-01T09:47:26.082 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.073+0000 7fe27ac3d700 1 -- 172.21.15.191:0/1086216950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740b2370 msgr2=0x7fe274160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.082 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.073+0000 7fe27ac3d700 1 --2- 172.21.15.191:0/1086216950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740b2370 0x7fe274160150 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe27406e540 tx=0x7fe260009bd0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.082 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.073+0000 7fe27ac3d700 1 -- 172.21.15.191:0/1086216950 shutdown_connections 2022-12-01T09:47:26.083 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.073+0000 7fe27ac3d700 1 --2- 172.21.15.191:0/1086216950 >> v2:172.21.15.84:3300/0 conn(0x7fe2740aef30 0x7fe27406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.083 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.073+0000 7fe27ac3d700 1 --2- 172.21.15.191:0/1086216950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740b2370 0x7fe274160150 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.083 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.073+0000 7fe27ac3d700 1 -- 172.21.15.191:0/1086216950 >> 172.21.15.191:0/1086216950 conn(0x7fe2740cff30 msgr2=0x7fe2740d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:26.084 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.073+0000 7fe27ac3d700 1 -- 172.21.15.191:0/1086216950 shutdown_connections 2022-12-01T09:47:26.084 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.073+0000 7fe27ac3d700 1 -- 172.21.15.191:0/1086216950 wait complete. 2022-12-01T09:47:26.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.075+0000 7fe27ac3d700 1 Processor -- start 2022-12-01T09:47:26.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.075+0000 7fe27ac3d700 1 -- start start 2022-12-01T09:47:26.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe27ac3d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740aef30 0x7fe27415e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.088 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe27ac3d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2740b2370 0x7fe27415e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe27ac3d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 0x7fe2740b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe27ac3d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe274156d70 con 0x7fe2740b2370 2022-12-01T09:47:26.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe27ac3d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2741e65d0 con 0x7fe2740aef30 2022-12-01T09:47:26.089 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe27ac3d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2741e68b0 con 0x7fe2740cbc90 2022-12-01T09:47:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740aef30 0x7fe27415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740aef30 0x7fe27415e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:47492/0 (socket says 172.21.15.191:47492) 2022-12-01T09:47:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 -- 172.21.15.191:0/2045267052 learned_addr learned my addr 172.21.15.191:0/2045267052 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2791da700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 0x7fe2740b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.090 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe273fff700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2740b2370 0x7fe27415e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2791da700 1 -- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 msgr2=0x7fe2740b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2022-12-01T09:47:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 -- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 msgr2=0x7fe2740b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2791da700 1 -- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 msgr2=0x7fe2740b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:26.091 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2791da700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 0x7fe2740b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2791da700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 0x7fe2740b3db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 0x7fe2740b3db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 -- 172.21.15.191:0/2045267052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2740b2370 msgr2=0x7fe27415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.092 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2740b2370 0x7fe27415e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 -- 172.21.15.191:0/2045267052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe260009550 con 0x7fe2740aef30 2022-12-01T09:47:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.076+0000 7fe2789d9700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740aef30 0x7fe27415e300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe26800e9b0 tx=0x7fe26800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.077+0000 7fe2717fa700 1 -- 172.21.15.191:0/2045267052 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268009710 con 0x7fe2740aef30 2022-12-01T09:47:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.077+0000 7fe2717fa700 1 -- 172.21.15.191:0/2045267052 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26800c850 con 0x7fe2740aef30 2022-12-01T09:47:26.093 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.077+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2741e6b90 con 0x7fe2740aef30 2022-12-01T09:47:26.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.077+0000 7fe2717fa700 1 -- 172.21.15.191:0/2045267052 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26800cbc0 con 0x7fe2740aef30 2022-12-01T09:47:26.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.077+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2741e7150 con 0x7fe2740aef30 2022-12-01T09:47:26.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.077+0000 7fe2717fa700 1 -- 172.21.15.191:0/2045267052 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe268019030 con 0x7fe2740aef30 2022-12-01T09:47:26.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.077+0000 7fe2717fa700 1 -- 172.21.15.191:0/2045267052 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fe2680067f0 con 0x7fe2740aef30 2022-12-01T09:47:26.094 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.078+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2740712c0 con 0x7fe2740aef30 2022-12-01T09:47:26.095 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.080+0000 7fe2717fa700 1 -- 172.21.15.191:0/2045267052 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fe268021060 con 0x7fe2740aef30 2022-12-01T09:47:26.101 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:26.098+0000 7f86607ad200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-12-01T09:47:26.166 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.162+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e70b8522-3456-479c-81c9-10e06ad3172f", "id": 6} v 0) v1 -- 0x7fe274067420 con 0x7fe2740aef30 2022-12-01T09:47:26.177 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.173+0000 7fe2717fa700 1 -- 172.21.15.191:0/2045267052 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e70b8522-3456-479c-81c9-10e06ad3172f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe268017070 con 0x7fe2740aef30 2022-12-01T09:47:26.177 INFO:teuthology.orchestra.run.smithi191.stdout:6 2022-12-01T09:47:26.178 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.174+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740aef30 msgr2=0x7fe27415e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.179 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.174+0000 7fe27ac3d700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740aef30 0x7fe27415e300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe26800e9b0 tx=0x7fe26800ecc0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.179 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.174+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 shutdown_connections 2022-12-01T09:47:26.179 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.174+0000 7fe27ac3d700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2740cbc90 0x7fe2740b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.174+0000 7fe27ac3d700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2740aef30 0x7fe27415e300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.174+0000 7fe27ac3d700 1 --2- 172.21.15.191:0/2045267052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2740b2370 0x7fe27415e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.174+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 >> 172.21.15.191:0/2045267052 conn(0x7fe2740cff30 msgr2=0x7fe27415a930 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:26.180 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.174+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 shutdown_connections 2022-12-01T09:47:26.181 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.175+0000 7fe27ac3d700 1 -- 172.21.15.191:0/2045267052 wait complete. 2022-12-01T09:47:26.182 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:26.179+0000 7f0b7d114200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-12-01T09:47:26.191 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new d2c33094-6303-4ed3-963c-e010df6aa820 7 2022-12-01T09:47:26.264 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:26.261+0000 7f86607ad200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-12-01T09:47:26.313 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.310+0000 7fcfbb11a700 1 Processor -- start 2022-12-01T09:47:26.314 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.310+0000 7fcfbb11a700 1 -- start start 2022-12-01T09:47:26.321 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfbb11a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcfb40ad990 0x7fcfb415be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.322 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfbb11a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 0x7fcfb415fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.322 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfbb11a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb415f820 con 0x7fcfb41601f0 2022-12-01T09:47:26.323 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfbb11a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb40add60 con 0x7fcfb40ad990 2022-12-01T09:47:26.323 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfbb11a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb415f4d0 con 0x7fcfb40ce420 2022-12-01T09:47:26.324 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 0x7fcfb415fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.324 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb8eb6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcfb40ad990 0x7fcfb415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.325 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb96b7700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fcfb41601f0 0x7fcfb40cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:53442/0 (socket says 172.21.15.191:53442) 2022-12-01T09:47:26.325 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb96b7700 1 -- 172.21.15.191:0/1536948955 learned_addr learned my addr 172.21.15.191:0/1536948955 (peer_addr_for_me v1:172.21.15.191:0/0) 2022-12-01T09:47:26.326 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 0x7fcfb415fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:37292/0 (socket says 172.21.15.191:37292) 2022-12-01T09:47:26.326 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb3fff700 1 -- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 msgr2=0x7fcfb415fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:47:26.327 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb3fff700 1 -- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 msgr2=0x7fcfb415fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:26.327 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb3fff700 1 --2- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 0x7fcfb415fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:26.328 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb3fff700 1 --2- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 0x7fcfb415fcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:26.328 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb8eb6700 1 -- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 msgr2=0x7fcfb415fcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:26.329 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb8eb6700 1 --2- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 0x7fcfb415fcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.329 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.311+0000 7fcfb8eb6700 1 -- 172.21.15.191:0/1536948955 >> v1:172.21.15.191:6789/0 conn(0x7fcfb41601f0 legacy=0x7fcfb40cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.330 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.312+0000 7fcfb8eb6700 1 -- 172.21.15.191:0/1536948955 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb415c730 con 0x7fcfb40ad990 2022-12-01T09:47:26.330 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.312+0000 7fcfb37fe700 1 -- 172.21.15.191:0/1536948955 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251837367 0 0) 0x7fcfb415f820 con 0x7fcfb41601f0 2022-12-01T09:47:26.330 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.312+0000 7fcfb8eb6700 1 --2- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb40ad990 0x7fcfb415be90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009820 tx=0x7fcfa8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10505c33d69fb2 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:26.331 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.312+0000 7fcfb37fe700 1 -- 172.21.15.191:0/1536948955 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8005e30 con 0x7fcfb40ad990 2022-12-01T09:47:26.331 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.312+0000 7fcfb37fe700 1 -- 172.21.15.191:0/1536948955 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8006030 con 0x7fcfb40ad990 2022-12-01T09:47:26.332 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.312+0000 7fcfb37fe700 1 -- 172.21.15.191:0/1536948955 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80063e0 con 0x7fcfb40ad990 2022-12-01T09:47:26.332 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.312+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb40ad990 msgr2=0x7fcfb415be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.332 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.312+0000 7fcfbb11a700 1 --2- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb40ad990 0x7fcfb415be90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009820 tx=0x7fcfa8009b30 comp rx=0 tx=0).stop 2022-12-01T09:47:26.332 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.313+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1536948955 shutdown_connections 2022-12-01T09:47:26.333 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.313+0000 7fcfbb11a700 1 --2- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb40ad990 0x7fcfb415be90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.333 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.313+0000 7fcfbb11a700 1 --2- 172.21.15.191:0/1536948955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ce420 0x7fcfb415fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.333 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.313+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1536948955 >> 172.21.15.191:0/1536948955 conn(0x7fcfb4070a30 msgr2=0x7fcfb4070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:26.333 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.313+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1536948955 shutdown_connections 2022-12-01T09:47:26.333 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.313+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1536948955 wait complete. 2022-12-01T09:47:26.334 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.314+0000 7fcfbb11a700 1 Processor -- start 2022-12-01T09:47:26.334 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.314+0000 7fcfbb11a700 1 -- start start 2022-12-01T09:47:26.334 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfbb11a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 0x7fcfb415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.334 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfbb11a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb40ce420 0x7fcfb4158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.335 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfbb11a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb41601f0 0x7fcfb40cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.335 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfbb11a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c6ad0 con 0x7fcfb40ce420 2022-12-01T09:47:26.335 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfbb11a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb41f1540 con 0x7fcfb41601f0 2022-12-01T09:47:26.335 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfbb11a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb41f1820 con 0x7fcfb40ad990 2022-12-01T09:47:26.335 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb41601f0 0x7fcfb40cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.336 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb8eb6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 0x7fcfb415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.336 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb41601f0 0x7fcfb40cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:47508/0 (socket says 172.21.15.191:47508) 2022-12-01T09:47:26.336 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 -- 172.21.15.191:0/1780014807 learned_addr learned my addr 172.21.15.191:0/1780014807 (peer_addr_for_me v2:172.21.15.191:0/0) 2022-12-01T09:47:26.336 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb8eb6700 1 -- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 msgr2=0x7fcfb415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-12-01T09:47:26.337 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb8eb6700 1 -- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 msgr2=0x7fcfb415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:26.338 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb8eb6700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 0x7fcfb415eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:26.338 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb3fff700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb40ce420 0x7fcfb4158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.338 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb8eb6700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 0x7fcfb415eac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:26.339 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 -- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 msgr2=0x7fcfb415eac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:26.339 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 0x7fcfb415eac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.339 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 -- 172.21.15.191:0/1780014807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb40ce420 msgr2=0x7fcfb4158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.340 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb40ce420 0x7fcfb4158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.340 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 -- 172.21.15.191:0/1780014807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8009430 con 0x7fcfb41601f0 2022-12-01T09:47:26.340 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.315+0000 7fcfb96b7700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb41601f0 0x7fcfb40cc030 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00cb40 tx=0x7fcfac00ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:26.341 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.316+0000 7fcfb17fa700 1 -- 172.21.15.191:0/1780014807 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac00aaa0 con 0x7fcfb41601f0 2022-12-01T09:47:26.341 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.316+0000 7fcfb17fa700 1 -- 172.21.15.191:0/1780014807 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac00ac20 con 0x7fcfb41601f0 2022-12-01T09:47:26.342 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.316+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb41f1b00 con 0x7fcfb41601f0 2022-12-01T09:47:26.342 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.316+0000 7fcfb17fa700 1 -- 172.21.15.191:0/1780014807 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0070d0 con 0x7fcfb41601f0 2022-12-01T09:47:26.342 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.316+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb41588c0 con 0x7fcfb41601f0 2022-12-01T09:47:26.343 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.316+0000 7fcfb17fa700 1 -- 172.21.15.191:0/1780014807 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fcfac01a030 con 0x7fcfb41601f0 2022-12-01T09:47:26.343 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.317+0000 7fcfb17fa700 1 -- 172.21.15.191:0/1780014807 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fcfac01ca10 con 0x7fcfb41601f0 2022-12-01T09:47:26.343 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.316+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb41f2020 con 0x7fcfb41601f0 2022-12-01T09:47:26.344 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.319+0000 7fcfb17fa700 1 -- 172.21.15.191:0/1780014807 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fcfb41f2020 con 0x7fcfb41601f0 2022-12-01T09:47:26.405 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.402+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d2c33094-6303-4ed3-963c-e010df6aa820", "id": 7} v 0) v1 -- 0x7fcfb40c6c10 con 0x7fcfb41601f0 2022-12-01T09:47:26.412 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.409+0000 7fcfb17fa700 1 -- 172.21.15.191:0/1780014807 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d2c33094-6303-4ed3-963c-e010df6aa820", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcfb40c6c10 con 0x7fcfb41601f0 2022-12-01T09:47:26.413 INFO:teuthology.orchestra.run.smithi191.stdout:7 2022-12-01T09:47:26.414 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb41601f0 msgr2=0x7fcfb40cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.414 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb41601f0 0x7fcfb40cc030 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00cb40 tx=0x7fcfac00ce50 comp rx=0 tx=0).stop 2022-12-01T09:47:26.415 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 shutdown_connections 2022-12-01T09:47:26.415 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfb41601f0 0x7fcfb40cc030 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.415 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfb40ad990 0x7fcfb415eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 --2- 172.21.15.191:0/1780014807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfb40ce420 0x7fcfb4158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 >> 172.21.15.191:0/1780014807 conn(0x7fcfb4070a30 msgr2=0x7fcfb41e52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:26.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 shutdown_connections 2022-12-01T09:47:26.416 INFO:teuthology.orchestra.run.smithi191.stderr:2022-12-01T09:47:26.410+0000 7fcfbb11a700 1 -- 172.21.15.191:0/1780014807 wait complete. 2022-12-01T09:47:26.425 INFO:tasks.ceph.osd.0:Restarting daemon 2022-12-01T09:47:26.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-12-01T09:47:26.428 INFO:tasks.ceph.osd.0:Started 2022-12-01T09:47:26.429 INFO:tasks.ceph.osd.1:Restarting daemon 2022-12-01T09:47:26.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-12-01T09:47:26.431 INFO:tasks.ceph.osd.1:Started 2022-12-01T09:47:26.431 INFO:tasks.ceph.osd.2:Restarting daemon 2022-12-01T09:47:26.432 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-12-01T09:47:26.434 INFO:tasks.ceph.osd.2:Started 2022-12-01T09:47:26.434 INFO:tasks.ceph.osd.3:Restarting daemon 2022-12-01T09:47:26.435 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-12-01T09:47:26.437 INFO:tasks.ceph.osd.3:Started 2022-12-01T09:47:26.438 INFO:tasks.ceph.osd.4:Restarting daemon 2022-12-01T09:47:26.438 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-12-01T09:47:26.440 INFO:tasks.ceph.osd.4:Started 2022-12-01T09:47:26.440 INFO:tasks.ceph.osd.5:Restarting daemon 2022-12-01T09:47:26.441 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-12-01T09:47:26.443 INFO:tasks.ceph.osd.5:Started 2022-12-01T09:47:26.443 INFO:tasks.ceph.osd.6:Restarting daemon 2022-12-01T09:47:26.444 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-12-01T09:47:26.446 INFO:tasks.ceph.osd.6:Started 2022-12-01T09:47:26.446 INFO:tasks.ceph.osd.7:Restarting daemon 2022-12-01T09:47:26.447 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-12-01T09:47:26.449 INFO:tasks.ceph.osd.7:Started 2022-12-01T09:47:26.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-12-01T09:47:26.604 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.600+0000 7f4bc7ea6700 1 Processor -- start 2022-12-01T09:47:26.605 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.601+0000 7f4bc7ea6700 1 -- start start 2022-12-01T09:47:26.614 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.601+0000 7f4bc7ea6700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4bc0157fc0 0x7f4bc00cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.614 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.601+0000 7f4bc7ea6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4bc006bb30 0x7f4bc006bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.614 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.601+0000 7f4bc7ea6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 0x7f4bc00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.615 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.601+0000 7f4bc7ea6700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc006b6a0 con 0x7f4bc0157fc0 2022-12-01T09:47:26.615 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.601+0000 7f4bc7ea6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc006b090 con 0x7f4bc006bb30 2022-12-01T09:47:26.616 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.601+0000 7f4bc7ea6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00715e0 con 0x7f4bc006c4e0 2022-12-01T09:47:26.616 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4bc0157fc0 0x7f4bc00cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.617 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4bc0157fc0 0x7f4bc00cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47370/0 (socket says 172.21.15.84:47370) 2022-12-01T09:47:26.617 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 -- 172.21.15.84:0/665732480 learned_addr learned my addr 172.21.15.84:0/665732480 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:26.617 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5441700 1 --2- 172.21.15.84:0/665732480 >> v2:172.21.15.191:3300/0 conn(0x7f4bc006bb30 0x7f4bc006bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.618 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc6443700 1 --2- 172.21.15.84:0/665732480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 0x7f4bc00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.618 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc6443700 1 -- 172.21.15.84:0/665732480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 msgr2=0x7f4bc00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-12-01T09:47:26.619 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc6443700 1 -- 172.21.15.84:0/665732480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 msgr2=0x7f4bc00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:26.620 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc6443700 1 --2- 172.21.15.84:0/665732480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 0x7f4bc00caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:26.620 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc6443700 1 --2- 172.21.15.84:0/665732480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 0x7f4bc00caea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:26.620 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 -- 172.21.15.84:0/665732480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 msgr2=0x7f4bc00caea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 --2- 172.21.15.84:0/665732480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 0x7f4bc00caea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 -- 172.21.15.84:0/665732480 >> v2:172.21.15.191:3300/0 conn(0x7f4bc006bb30 msgr2=0x7f4bc006bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 --2- 172.21.15.84:0/665732480 >> v2:172.21.15.191:3300/0 conn(0x7f4bc006bb30 0x7f4bc006bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.621 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 -- 172.21.15.84:0/665732480 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc006b330 con 0x7f4bc0157fc0 2022-12-01T09:47:26.622 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc5c42700 1 --2- 172.21.15.84:0/665732480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc00cc090 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4bc006e0b0 tx=0x7f4bb000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dd6b8638096a5e server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:26.622 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc4c40700 1 -- 172.21.15.84:0/665732480 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0005e30 con 0x7f4bc0157fc0 2022-12-01T09:47:26.622 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc4c40700 1 -- 172.21.15.84:0/665732480 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb0004030 con 0x7f4bc0157fc0 2022-12-01T09:47:26.623 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.602+0000 7f4bc4c40700 1 -- 172.21.15.84:0/665732480 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00043e0 con 0x7f4bc0157fc0 2022-12-01T09:47:26.623 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/665732480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 msgr2=0x7f4bc00cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.623 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 --2- 172.21.15.84:0/665732480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc00cc090 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4bc006e0b0 tx=0x7f4bb000be70 comp rx=0 tx=0).stop 2022-12-01T09:47:26.624 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/665732480 shutdown_connections 2022-12-01T09:47:26.624 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 --2- 172.21.15.84:0/665732480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006c4e0 0x7f4bc00caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.624 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 --2- 172.21.15.84:0/665732480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc00cc090 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.624 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 --2- 172.21.15.84:0/665732480 >> v2:172.21.15.191:3300/0 conn(0x7f4bc006bb30 0x7f4bc006bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.625 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/665732480 >> 172.21.15.84:0/665732480 conn(0x7f4bc0070a30 msgr2=0x7f4bc0070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:26.625 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/665732480 shutdown_connections 2022-12-01T09:47:26.625 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.603+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/665732480 wait complete. 2022-12-01T09:47:26.625 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.604+0000 7f4bc7ea6700 1 Processor -- start 2022-12-01T09:47:26.625 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.604+0000 7f4bc7ea6700 1 -- start start 2022-12-01T09:47:26.626 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc7ea6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 0x7f4bc01f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.626 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc7ea6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bc006c4e0 0x7f4bc009b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.626 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc7ea6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc0064ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:26.626 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc7ea6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc009a6c0 con 0x7f4bc0157fc0 2022-12-01T09:47:26.627 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc7ea6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc009a9a0 con 0x7f4bc006c4e0 2022-12-01T09:47:26.627 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc7ea6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc009ac80 con 0x7f4bc006bb30 2022-12-01T09:47:26.627 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc0064ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.627 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc0064ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47380/0 (socket says 172.21.15.84:47380) 2022-12-01T09:47:26.628 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 -- 172.21.15.84:0/1927323251 learned_addr learned my addr 172.21.15.84:0/1927323251 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:26.628 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc5441700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bc006c4e0 0x7f4bc009b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.628 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc5c42700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 0x7f4bc01f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:26.628 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc5c42700 1 -- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 msgr2=0x7f4bc01f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2022-12-01T09:47:26.629 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc5c42700 1 -- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 msgr2=0x7f4bc01f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:26.629 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc5c42700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 0x7f4bc01f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:26.629 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc5c42700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 0x7f4bc01f4050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:26.629 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 -- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 msgr2=0x7f4bc01f4050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:26.630 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 0x7f4bc01f4050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.630 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 -- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bc006c4e0 msgr2=0x7f4bc009b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.630 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bc006c4e0 0x7f4bc009b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.630 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 -- 172.21.15.84:0/1927323251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb000b7a0 con 0x7f4bc0157fc0 2022-12-01T09:47:26.631 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.605+0000 7f4bc6443700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc0064ec0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80070b0 tx=0x7f4bb800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:26.631 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.606+0000 7f4bae7fc700 1 -- 172.21.15.84:0/1927323251 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800ede0 con 0x7f4bc0157fc0 2022-12-01T09:47:26.631 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.606+0000 7f4bae7fc700 1 -- 172.21.15.84:0/1927323251 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb8014420 con 0x7f4bc0157fc0 2022-12-01T09:47:26.631 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.606+0000 7f4bae7fc700 1 -- 172.21.15.84:0/1927323251 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80147d0 con 0x7f4bc0157fc0 2022-12-01T09:47:26.632 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.606+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc00aceb0 con 0x7f4bc0157fc0 2022-12-01T09:47:26.632 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.606+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc009a350 con 0x7f4bc0157fc0 2022-12-01T09:47:26.632 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.606+0000 7f4bae7fc700 1 -- 172.21.15.84:0/1927323251 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4bb8012050 con 0x7f4bc0157fc0 2022-12-01T09:47:26.633 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.606+0000 7f4bae7fc700 1 -- 172.21.15.84:0/1927323251 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f4bb801e460 con 0x7f4bc0157fc0 2022-12-01T09:47:26.633 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.607+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc0067420 con 0x7f4bc0157fc0 2022-12-01T09:47:26.633 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.609+0000 7f4bae7fc700 1 -- 172.21.15.84:0/1927323251 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f4bb8014c20 con 0x7f4bc0157fc0 2022-12-01T09:47:26.681 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.678+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bc00c8d40 con 0x7f4bc0157fc0 2022-12-01T09:47:26.682 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.678+0000 7f4bae7fc700 1 -- 172.21.15.84:0/1927323251 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f4bb8014dc0 con 0x7f4bc0157fc0 2022-12-01T09:47:26.682 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:47:26.682 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":10,"fsid":"8329cc4d-9cf5-499b-b4c9-a9793681da24","created":"2022-12-01T09:47:23.909978+0000","modified":"2022-12-01T09:47:26.403178+0000","last_up_change":"0.000000","last_in_change":"2022-12-01T09:47:26.403178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"e9c852ba-df6e-4c09-98f8-692b089e63c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b47da52-0666-49d8-b263-725119e42db4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b668a53-55d9-481c-83a5-df590cd91fad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1f2c87e3-7398-4f50-a56c-5f798d2fb279","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8a4b43a4-6c63-48ab-826a-cb8919c44b8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"18a3160f-c761-49e7-8d7d-56fdf4c8a7f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e70b8522-3456-479c-81c9-10e06ad3172f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d2c33094-6303-4ed3-963c-e010df6aa820","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:47:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 msgr2=0x7f4bc0064ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc0064ec0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80070b0 tx=0x7f4bb800e3c0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 shutdown_connections 2022-12-01T09:47:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4bc0157fc0 0x7f4bc0064ec0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4bc006bb30 0x7f4bc01f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 --2- 172.21.15.84:0/1927323251 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bc006c4e0 0x7f4bc009b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 >> 172.21.15.84:0/1927323251 conn(0x7f4bc0070a30 msgr2=0x7f4bc00ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 shutdown_connections 2022-12-01T09:47:26.686 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:26.680+0000 7f4bc7ea6700 1 -- 172.21.15.84:0/1927323251 wait complete. 2022-12-01T09:47:26.698 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-12-01T09:47:26.699 INFO:tasks.ceph:Waiting for OSDs to come up 2022-12-01T09:47:26.894 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:26.890+0000 7f0b7d114200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-12-01T09:47:26.969 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:26.966+0000 7f86607ad200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-12-01T09:47:26.970 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:26.967+0000 7f0b7d114200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-12-01T09:47:27.039 INFO:tasks.ceph.osd.4.smithi191.stderr:2022-12-01T09:47:27.035+0000 7f8706a575c0 -1 Falling back to public interface 2022-12-01T09:47:27.044 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:27.041+0000 7f86607ad200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-12-01T09:47:27.050 INFO:tasks.ceph.osd.2.smithi084.stderr:2022-12-01T09:47:27.047+0000 7f19d22e55c0 -1 Falling back to public interface 2022-12-01T09:47:27.051 INFO:tasks.ceph.osd.0.smithi084.stderr:2022-12-01T09:47:27.047+0000 7fe83e18f5c0 -1 Falling back to public interface 2022-12-01T09:47:27.051 INFO:tasks.ceph.osd.0.smithi084.stderr:2022-12-01T09:47:27.047+0000 7fe83554d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2022-12-01T09:47:27.052 INFO:tasks.ceph.osd.6.smithi191.stderr:2022-12-01T09:47:27.047+0000 7fc26dfb75c0 -1 Falling back to public interface 2022-12-01T09:47:27.060 INFO:tasks.ceph.osd.3.smithi084.stderr:2022-12-01T09:47:27.057+0000 7f38c76765c0 -1 Falling back to public interface 2022-12-01T09:47:27.064 INFO:tasks.ceph.osd.1.smithi084.stderr:2022-12-01T09:47:27.061+0000 7f6e321bd5c0 -1 Falling back to public interface 2022-12-01T09:47:27.067 INFO:tasks.ceph.osd.5.smithi191.stderr:2022-12-01T09:47:27.064+0000 7fe87fdf15c0 -1 Falling back to public interface 2022-12-01T09:47:27.072 INFO:tasks.ceph.osd.7.smithi191.stderr:2022-12-01T09:47:27.069+0000 7f42051cb5c0 -1 Falling back to public interface 2022-12-01T09:47:27.111 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:27.108+0000 7f0b7d114200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-12-01T09:47:27.184 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:27.181+0000 7f86607ad200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-12-01T09:47:27.268 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:27.265+0000 7f0b7d114200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-12-01T09:47:27.342 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:27.339+0000 7f86607ad200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-12-01T09:47:27.400 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:27.397+0000 7f0b7d114200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-12-01T09:47:27.473 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:27.470+0000 7f86607ad200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-12-01T09:47:27.504 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-12-01T09:47:27.629 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.615+0000 7f82b43c6700 1 Processor -- start 2022-12-01T09:47:27.629 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.615+0000 7f82b43c6700 1 -- start start 2022-12-01T09:47:27.630 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b43c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 0x7f82ac069dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:27.630 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b43c6700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82ac0ae140 0x7f82ac06a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:27.630 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b43c6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ac06c280 con 0x7f82ac160290 2022-12-01T09:47:27.630 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b43c6700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac0cde00 con 0x7f82ac0ae140 2022-12-01T09:47:27.631 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b43c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0ce0e0 con 0x7f82ac072670 2022-12-01T09:47:27.631 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b2963700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82ac0ae140 0x7f82ac06a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:27.631 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b2963700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f82ac0ae140 0x7f82ac06a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47386/0 (socket says 172.21.15.84:47386) 2022-12-01T09:47:27.631 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b2963700 1 -- 172.21.15.84:0/2517060301 learned_addr learned my addr 172.21.15.84:0/2517060301 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:27.632 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b1961700 1 --2- 172.21.15.84:0/2517060301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 0x7f82ac069dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:27.632 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b1961700 1 -- 172.21.15.84:0/2517060301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 msgr2=0x7f82ac069dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-12-01T09:47:27.632 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b1961700 1 -- 172.21.15.84:0/2517060301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 msgr2=0x7f82ac069dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-12-01T09:47:27.632 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b1961700 1 --2- 172.21.15.84:0/2517060301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 0x7f82ac069dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:27.633 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b1961700 1 --2- 172.21.15.84:0/2517060301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 0x7f82ac069dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:27.633 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b2963700 1 -- 172.21.15.84:0/2517060301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 msgr2=0x7f82ac069dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-12-01T09:47:27.633 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b2963700 1 --2- 172.21.15.84:0/2517060301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 0x7f82ac069dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.633 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b2963700 1 -- 172.21.15.84:0/2517060301 >> v1:172.21.15.191:6789/0 conn(0x7f82ac160290 legacy=0x7f82ac071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:27.633 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b2963700 1 -- 172.21.15.84:0/2517060301 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac0ce3c0 con 0x7f82ac0ae140 2022-12-01T09:47:27.634 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.616+0000 7f82b2963700 1 --2- 172.21.15.84:0/2517060301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac0ae140 0x7f82ac06a300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009e50 tx=0x7f82a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b0dcd90e259cfa server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:27.634 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b1160700 1 -- 172.21.15.84:0/2517060301 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a400e070 con 0x7f82ac0ae140 2022-12-01T09:47:27.634 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b1160700 1 -- 172.21.15.84:0/2517060301 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a400b9c0 con 0x7f82ac0ae140 2022-12-01T09:47:27.635 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b1160700 1 -- 172.21.15.84:0/2517060301 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a400bd70 con 0x7f82ac0ae140 2022-12-01T09:47:27.635 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b43c6700 1 -- 172.21.15.84:0/2517060301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac0ae140 msgr2=0x7f82ac06a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:27.635 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b43c6700 1 --2- 172.21.15.84:0/2517060301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac0ae140 0x7f82ac06a300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009e50 tx=0x7f82a400b330 comp rx=0 tx=0).stop 2022-12-01T09:47:27.635 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b43c6700 1 -- 172.21.15.84:0/2517060301 shutdown_connections 2022-12-01T09:47:27.636 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b43c6700 1 --2- 172.21.15.84:0/2517060301 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac0ae140 0x7f82ac06a300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.636 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b43c6700 1 --2- 172.21.15.84:0/2517060301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac072670 0x7f82ac069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.636 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b43c6700 1 -- 172.21.15.84:0/2517060301 >> 172.21.15.84:0/2517060301 conn(0x7f82ac0b1640 msgr2=0x7f82ac0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:27.636 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b43c6700 1 -- 172.21.15.84:0/2517060301 shutdown_connections 2022-12-01T09:47:27.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.617+0000 7f82b43c6700 1 -- 172.21.15.84:0/2517060301 wait complete. 2022-12-01T09:47:27.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.618+0000 7f82b43c6700 1 Processor -- start 2022-12-01T09:47:27.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.618+0000 7f82b43c6700 1 -- start start 2022-12-01T09:47:27.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b43c6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82ac072670 0x7f82ac15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:27.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b43c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 0x7f82ac0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:27.638 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b43c6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac160290 0x7f82ac0abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:27.638 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b43c6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0cb3a0 con 0x7f82ac160290 2022-12-01T09:47:27.638 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b43c6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0c6b60 con 0x7f82ac072670 2022-12-01T09:47:27.638 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b43c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0c92e0 con 0x7f82ac0ae140 2022-12-01T09:47:27.639 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b2963700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac160290 0x7f82ac0abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:27.639 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b2963700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac160290 0x7f82ac0abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47390/0 (socket says 172.21.15.84:47390) 2022-12-01T09:47:27.639 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b2963700 1 -- 172.21.15.84:0/221563427 learned_addr learned my addr 172.21.15.84:0/221563427 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:27.639 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b2162700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82ac072670 0x7f82ac15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:27.640 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.619+0000 7f82b1961700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 0x7f82ac0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:27.640 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b1961700 1 -- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 msgr2=0x7f82ac0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2022-12-01T09:47:27.640 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b1961700 1 -- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 msgr2=0x7f82ac0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2022-12-01T09:47:27.641 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b1961700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 0x7f82ac0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-12-01T09:47:27.641 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b1961700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 0x7f82ac0c9dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-12-01T09:47:27.641 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b2963700 1 -- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 msgr2=0x7f82ac0c9dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2022-12-01T09:47:27.642 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b2963700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 0x7f82ac0c9dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.642 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b2963700 1 -- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82ac072670 msgr2=0x7f82ac15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:27.642 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b2963700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82ac072670 0x7f82ac15a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.642 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b2963700 1 -- 172.21.15.84:0/221563427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4009b30 con 0x7f82ac160290 2022-12-01T09:47:27.643 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b2963700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac160290 0x7f82ac0abc20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82a4012000 tx=0x7f82a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:27.643 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f829affd700 1 -- 172.21.15.84:0/221563427 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40048f0 con 0x7f82ac160290 2022-12-01T09:47:27.643 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f829affd700 1 -- 172.21.15.84:0/221563427 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a40099b0 con 0x7f82ac160290 2022-12-01T09:47:27.643 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f829affd700 1 -- 172.21.15.84:0/221563427 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a401c680 con 0x7f82ac160290 2022-12-01T09:47:27.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac0c95c0 con 0x7f82ac160290 2022-12-01T09:47:27.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac15a930 con 0x7f82ac160290 2022-12-01T09:47:27.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f829affd700 1 -- 172.21.15.84:0/221563427 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f82a4022030 con 0x7f82ac160290 2022-12-01T09:47:27.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.620+0000 7f829affd700 1 -- 172.21.15.84:0/221563427 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f82a4012ac0 con 0x7f82ac160290 2022-12-01T09:47:27.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.621+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac0c9140 con 0x7f82ac160290 2022-12-01T09:47:27.645 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.625+0000 7f829affd700 1 -- 172.21.15.84:0/221563427 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f82ac0c9140 con 0x7f82ac160290 2022-12-01T09:47:27.699 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.695+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82ac067420 con 0x7f82ac160290 2022-12-01T09:47:27.700 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.695+0000 7f829affd700 1 -- 172.21.15.84:0/221563427 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f82a4020070 con 0x7f82ac160290 2022-12-01T09:47:27.701 INFO:teuthology.misc.health.smithi084.stdout: 2022-12-01T09:47:27.702 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":10,"fsid":"8329cc4d-9cf5-499b-b4c9-a9793681da24","created":"2022-12-01T09:47:23.909978+0000","modified":"2022-12-01T09:47:26.403178+0000","last_up_change":"0.000000","last_in_change":"2022-12-01T09:47:26.403178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"e9c852ba-df6e-4c09-98f8-692b089e63c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b47da52-0666-49d8-b263-725119e42db4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b668a53-55d9-481c-83a5-df590cd91fad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1f2c87e3-7398-4f50-a56c-5f798d2fb279","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8a4b43a4-6c63-48ab-826a-cb8919c44b8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"18a3160f-c761-49e7-8d7d-56fdf4c8a7f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e70b8522-3456-479c-81c9-10e06ad3172f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d2c33094-6303-4ed3-963c-e010df6aa820","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:47:27.704 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac160290 msgr2=0x7f82ac0abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:27.705 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac160290 0x7f82ac0abc20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82a4012000 tx=0x7f82a4004ad0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.706 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 shutdown_connections 2022-12-01T09:47:27.706 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82ac160290 0x7f82ac0abc20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.706 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82ac072670 0x7f82ac15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.706 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 --2- 172.21.15.84:0/221563427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82ac0ae140 0x7f82ac0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:27.706 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 >> 172.21.15.84:0/221563427 conn(0x7f82ac0b1640 msgr2=0x7f82ac0ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:27.707 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 shutdown_connections 2022-12-01T09:47:27.707 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:27.696+0000 7f82b43c6700 1 -- 172.21.15.84:0/221563427 wait complete. 2022-12-01T09:47:27.710 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-12-01T09:47:27.757 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:27.754+0000 7f0b7d114200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-12-01T09:47:27.830 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:27.826+0000 7f86607ad200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-12-01T09:47:27.908 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:27.905+0000 7f0b7d114200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-12-01T09:47:27.982 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:27.979+0000 7f86607ad200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-12-01T09:47:28.022 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:28.019+0000 7f0b7d114200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-12-01T09:47:28.094 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:28.091+0000 7f86607ad200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-12-01T09:47:28.166 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:28.162+0000 7f0b7d114200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-12-01T09:47:28.239 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:28.236+0000 7f86607ad200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-12-01T09:47:28.262 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:28.259+0000 7f0b7d114200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-12-01T09:47:28.336 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:28.332+0000 7f86607ad200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-12-01T09:47:28.610 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:28.607+0000 7f0b7d114200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-12-01T09:47:28.699 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:28.695+0000 7f86607ad200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-12-01T09:47:29.183 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:29.180+0000 7f0b7d114200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-12-01T09:47:29.280 INFO:tasks.ceph.mgr.x.smithi191.stderr:2022-12-01T09:47:29.275+0000 7f86607ad200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-12-01T09:47:30.211 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:30.208+0000 7f0b354c6700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-12-01T09:47:30.212 INFO:tasks.ceph.mgr.y.smithi084.stderr:2022-12-01T09:47:30.208+0000 7f0b354c6700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-12-01T09:47:30.542 INFO:tasks.ceph.osd.4.smithi191.stderr:2022-12-01T09:47:30.538+0000 7f8706a575c0 -1 osd.4 0 log_to_monitors true 2022-12-01T09:47:30.551 INFO:tasks.ceph.osd.6.smithi191.stderr:2022-12-01T09:47:30.547+0000 7fc26dfb75c0 -1 osd.6 0 log_to_monitors true 2022-12-01T09:47:30.569 INFO:tasks.ceph.osd.5.smithi191.stderr:2022-12-01T09:47:30.558+0000 7fe87fdf15c0 -1 osd.5 0 log_to_monitors true 2022-12-01T09:47:30.570 INFO:tasks.ceph.osd.2.smithi084.stderr:2022-12-01T09:47:30.560+0000 7f19d22e55c0 -1 osd.2 0 log_to_monitors true 2022-12-01T09:47:30.576 INFO:tasks.ceph.osd.3.smithi084.stderr:2022-12-01T09:47:30.573+0000 7f38c76765c0 -1 osd.3 0 log_to_monitors true 2022-12-01T09:47:30.580 INFO:tasks.ceph.osd.0.smithi084.stderr:2022-12-01T09:47:30.577+0000 7fe83e18f5c0 -1 osd.0 0 log_to_monitors true 2022-12-01T09:47:30.584 INFO:tasks.ceph.osd.1.smithi084.stderr:2022-12-01T09:47:30.581+0000 7f6e321bd5c0 -1 osd.1 0 log_to_monitors true 2022-12-01T09:47:30.595 INFO:tasks.ceph.osd.7.smithi191.stderr:2022-12-01T09:47:30.591+0000 7f42051cb5c0 -1 osd.7 0 log_to_monitors true 2022-12-01T09:47:34.176 INFO:tasks.ceph.osd.6.smithi191.stderr:2022-12-01T09:47:34.172+0000 7fc261b6e700 -1 osd.6 0 waiting for initial osdmap 2022-12-01T09:47:34.183 INFO:tasks.ceph.osd.2.smithi084.stderr:2022-12-01T09:47:34.180+0000 7f19c5e9c700 -1 osd.2 0 waiting for initial osdmap 2022-12-01T09:47:34.193 INFO:tasks.ceph.osd.1.smithi084.stderr:2022-12-01T09:47:34.190+0000 7f6e25d74700 -1 osd.1 0 waiting for initial osdmap 2022-12-01T09:47:34.196 INFO:tasks.ceph.osd.6.smithi191.stderr:2022-12-01T09:47:34.192+0000 7fc259f49700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:47:34.198 INFO:tasks.ceph.osd.2.smithi084.stderr:2022-12-01T09:47:34.195+0000 7f19be277700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:47:34.207 INFO:tasks.ceph.osd.1.smithi084.stderr:2022-12-01T09:47:34.204+0000 7f6e1e14f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:47:34.222 INFO:tasks.ceph.osd.5.smithi191.stderr:2022-12-01T09:47:34.218+0000 7fe8739a8700 -1 osd.5 0 waiting for initial osdmap 2022-12-01T09:47:34.235 INFO:tasks.ceph.osd.5.smithi191.stderr:2022-12-01T09:47:34.231+0000 7fe86bd83700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:47:34.245 INFO:tasks.ceph.osd.3.smithi084.stderr:2022-12-01T09:47:34.242+0000 7f38bb22d700 -1 osd.3 0 waiting for initial osdmap 2022-12-01T09:47:34.246 INFO:tasks.ceph.osd.0.smithi084.stderr:2022-12-01T09:47:34.242+0000 7fe831d46700 -1 osd.0 0 waiting for initial osdmap 2022-12-01T09:47:34.253 INFO:tasks.ceph.osd.4.smithi191.stderr:2022-12-01T09:47:34.249+0000 7f86fa60e700 -1 osd.4 0 waiting for initial osdmap 2022-12-01T09:47:34.258 INFO:tasks.ceph.osd.3.smithi084.stderr:2022-12-01T09:47:34.255+0000 7f38b3608700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:47:34.262 INFO:tasks.ceph.osd.0.smithi084.stderr:2022-12-01T09:47:34.259+0000 7fe82a121700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:47:34.267 INFO:tasks.ceph.osd.7.smithi191.stderr:2022-12-01T09:47:34.263+0000 7f41f8d82700 -1 osd.7 0 waiting for initial osdmap 2022-12-01T09:47:34.271 INFO:tasks.ceph.osd.4.smithi191.stderr:2022-12-01T09:47:34.267+0000 7f86f29e9700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:47:34.284 INFO:tasks.ceph.osd.7.smithi191.stderr:2022-12-01T09:47:34.280+0000 7f41f115d700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-12-01T09:47:34.517 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-12-01T09:47:34.629 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.625+0000 7fdd7dda0700 1 Processor -- start 2022-12-01T09:47:34.629 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.625+0000 7fdd7dda0700 1 -- start start 2022-12-01T09:47:34.636 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd7dda0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdd780701a0 0x7fdd7815cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:34.636 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd7dda0700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd781580d0 0x7fdd781584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:34.636 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd7dda0700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7815d100 con 0x7fdd78070570 2022-12-01T09:47:34.636 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd7dda0700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd780cd260 con 0x7fdd781580d0 2022-12-01T09:47:34.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd7dda0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd78069770 con 0x7fdd780701a0 2022-12-01T09:47:34.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd77fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd781580d0 0x7fdd781584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:34.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd77fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdd781580d0 0x7fdd781584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47498/0 (socket says 172.21.15.84:47498) 2022-12-01T09:47:34.637 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd76ffd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdd78070570 0x7fdd7806a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52346/0 (socket says 172.21.15.84:52346) 2022-12-01T09:47:34.638 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd77fff700 1 -- 172.21.15.84:0/1840485828 learned_addr learned my addr 172.21.15.84:0/1840485828 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:34.638 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.626+0000 7fdd777fe700 1 --2- 172.21.15.84:0/1840485828 >> v2:172.21.15.191:3300/0 conn(0x7fdd780701a0 0x7fdd7815cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:34.638 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd77fff700 1 -- 172.21.15.84:0/1840485828 >> v2:172.21.15.191:3300/0 conn(0x7fdd780701a0 msgr2=0x7fdd7815cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:34.638 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd77fff700 1 --2- 172.21.15.84:0/1840485828 >> v2:172.21.15.191:3300/0 conn(0x7fdd780701a0 0x7fdd7815cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.639 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd77fff700 1 -- 172.21.15.84:0/1840485828 >> v1:172.21.15.84:6789/0 conn(0x7fdd78070570 legacy=0x7fdd7806a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:34.639 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd767fc700 1 -- 172.21.15.84:0/1840485828 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217826534 0 0) 0x7fdd68001110 con 0x7fdd78070570 2022-12-01T09:47:34.639 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd77fff700 1 -- 172.21.15.84:0/1840485828 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78069a50 con 0x7fdd781580d0 2022-12-01T09:47:34.639 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd77fff700 1 --2- 172.21.15.84:0/1840485828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd781580d0 0x7fdd781584c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b1aa0 tx=0x7fdd7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c494836ad230d66a server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:34.640 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd767fc700 1 -- 172.21.15.84:0/1840485828 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000e070 con 0x7fdd781580d0 2022-12-01T09:47:34.640 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd767fc700 1 -- 172.21.15.84:0/1840485828 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7000b910 con 0x7fdd781580d0 2022-12-01T09:47:34.640 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd767fc700 1 -- 172.21.15.84:0/1840485828 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000bcc0 con 0x7fdd781580d0 2022-12-01T09:47:34.640 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1840485828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd781580d0 msgr2=0x7fdd781584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:34.641 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1840485828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd781580d0 0x7fdd781584c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdd780b1aa0 tx=0x7fdd7000b330 comp rx=0 tx=0).stop 2022-12-01T09:47:34.641 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1840485828 shutdown_connections 2022-12-01T09:47:34.641 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1840485828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd781580d0 0x7fdd781584c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.641 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1840485828 >> v2:172.21.15.191:3300/0 conn(0x7fdd780701a0 0x7fdd7815cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.642 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1840485828 >> 172.21.15.84:0/1840485828 conn(0x7fdd780b1640 msgr2=0x7fdd780cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:34.642 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1840485828 shutdown_connections 2022-12-01T09:47:34.642 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.627+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1840485828 wait complete. 2022-12-01T09:47:34.642 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.629+0000 7fdd7dda0700 1 Processor -- start 2022-12-01T09:47:34.643 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.629+0000 7fdd7dda0700 1 -- start start 2022-12-01T09:47:34.643 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.629+0000 7fdd7dda0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd780701a0 0x7fdd78149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:34.643 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.629+0000 7fdd7dda0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd78070570 0x7fdd78155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:34.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd7dda0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd781580d0 0x7fdd780c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:34.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd7dda0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78146360 con 0x7fdd78070570 2022-12-01T09:47:34.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd7dda0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd781476c0 con 0x7fdd781580d0 2022-12-01T09:47:34.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd7dda0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780cc370 con 0x7fdd780701a0 2022-12-01T09:47:34.644 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd78070570 0x7fdd78155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:34.645 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd78070570 0x7fdd78155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47502/0 (socket says 172.21.15.84:47502) 2022-12-01T09:47:34.645 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 -- 172.21.15.84:0/1615200779 learned_addr learned my addr 172.21.15.84:0/1615200779 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:34.645 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd77fff700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd781580d0 0x7fdd780c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:34.645 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd777fe700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd780701a0 0x7fdd78149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:34.646 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 -- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd780701a0 msgr2=0x7fdd78149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:34.646 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd780701a0 0x7fdd78149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.646 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 -- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd781580d0 msgr2=0x7fdd780c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:34.646 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd781580d0 0x7fdd780c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.647 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 -- 172.21.15.84:0/1615200779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70009b30 con 0x7fdd78070570 2022-12-01T09:47:34.647 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd77fff700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd781580d0 0x7fdd780c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:47:34.647 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd777fe700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd780701a0 0x7fdd78149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:47:34.647 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd76ffd700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd78070570 0x7fdd78155460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800c760 tx=0x7fdd6800cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:34.648 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd5ffff700 1 -- 172.21.15.84:0/1615200779 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68004c80 con 0x7fdd78070570 2022-12-01T09:47:34.648 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd5ffff700 1 -- 172.21.15.84:0/1615200779 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd68004e00 con 0x7fdd78070570 2022-12-01T09:47:34.648 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd781495d0 con 0x7fdd78070570 2022-12-01T09:47:34.648 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd5ffff700 1 -- 172.21.15.84:0/1615200779 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd68007310 con 0x7fdd78070570 2022-12-01T09:47:34.649 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.630+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78146d90 con 0x7fdd78070570 2022-12-01T09:47:34.649 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.631+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd780cfdb0 con 0x7fdd78070570 2022-12-01T09:47:34.649 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.631+0000 7fdd5ffff700 1 -- 172.21.15.84:0/1615200779 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7fdd6801a020 con 0x7fdd78070570 2022-12-01T09:47:34.649 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.632+0000 7fdd5ffff700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fdd58066540 0x7fdd580689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:34.649 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.632+0000 7fdd5ffff700 1 -- 172.21.15.84:0/1615200779 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fdd68082340 con 0x7fdd78070570 2022-12-01T09:47:34.650 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.632+0000 7fdd777fe700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fdd58066540 0x7fdd580689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:34.650 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.632+0000 7fdd777fe700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fdd58066540 0x7fdd580689f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009b60 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:34.651 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.636+0000 7fdd5ffff700 1 -- 172.21.15.84:0/1615200779 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7fdd68050990 con 0x7fdd78070570 2022-12-01T09:47:34.819 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.815+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd78067420 con 0x7fdd78070570 2022-12-01T09:47:34.820 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.815+0000 7fdd5ffff700 1 -- 172.21.15.84:0/1615200779 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7011 (secure 0 0 0) 0x7fdd68053fb0 con 0x7fdd78070570 2022-12-01T09:47:34.821 INFO:teuthology.misc.health.smithi084.stdout: 2022-12-01T09:47:34.821 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":12,"fsid":"8329cc4d-9cf5-499b-b4c9-a9793681da24","created":"2022-12-01T09:47:23.909978+0000","modified":"2022-12-01T09:47:32.211317+0000","last_up_change":"0.000000","last_in_change":"2022-12-01T09:47:26.403178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"e9c852ba-df6e-4c09-98f8-692b089e63c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b47da52-0666-49d8-b263-725119e42db4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7b668a53-55d9-481c-83a5-df590cd91fad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1f2c87e3-7398-4f50-a56c-5f798d2fb279","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8a4b43a4-6c63-48ab-826a-cb8919c44b8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"18a3160f-c761-49e7-8d7d-56fdf4c8a7f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e70b8522-3456-479c-81c9-10e06ad3172f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d2c33094-6303-4ed3-963c-e010df6aa820","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:47:34.822 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.817+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fdd58066540 msgr2=0x7fdd580689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:34.822 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.817+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fdd58066540 0x7fdd580689f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009b60 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2022-12-01T09:47:34.823 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.817+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd78070570 msgr2=0x7fdd78155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:34.823 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.817+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd78070570 0x7fdd78155460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800c760 tx=0x7fdd6800cb20 comp rx=0 tx=0).stop 2022-12-01T09:47:34.823 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.818+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 shutdown_connections 2022-12-01T09:47:34.823 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.818+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd781580d0 0x7fdd780c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.824 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.818+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd780701a0 0x7fdd78149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.824 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.818+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fdd58066540 0x7fdd580689f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.824 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.818+0000 7fdd7dda0700 1 --2- 172.21.15.84:0/1615200779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd78070570 0x7fdd78155460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:34.824 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.818+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 >> 172.21.15.84:0/1615200779 conn(0x7fdd780b1640 msgr2=0x7fdd780b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:34.825 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.818+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 shutdown_connections 2022-12-01T09:47:34.825 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:34.818+0000 7fdd7dda0700 1 -- 172.21.15.84:0/1615200779 wait complete. 2022-12-01T09:47:34.833 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-12-01T09:47:41.641 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-12-01T09:47:41.764 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.760+0000 7f988fa0c700 1 Processor -- start 2022-12-01T09:47:41.765 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.760+0000 7f988fa0c700 1 -- start start 2022-12-01T09:47:41.768 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.761+0000 7f988fa0c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f988815b950 0x7f98880cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:41.768 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.761+0000 7f988fa0c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f988815fc10 0x7f98880cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:41.768 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.761+0000 7f988fa0c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9888160270 con 0x7f98880cdc90 2022-12-01T09:47:41.769 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.761+0000 7f988fa0c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9888160070 con 0x7f988815b950 2022-12-01T09:47:41.769 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.761+0000 7f988fa0c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98880ce110 con 0x7f988815fc10 2022-12-01T09:47:41.769 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988d7a8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f988815b950 0x7f98880cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:41.769 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988dfa9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f988815fc10 0x7f98880cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:41.770 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988dfa9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f988815fc10 0x7f98880cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.84:45492/0 (socket says 172.21.15.84:45492) 2022-12-01T09:47:41.770 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988d7a8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f988815b950 0x7f98880cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.84:58020/0 (socket says 172.21.15.84:58020) 2022-12-01T09:47:41.770 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988cfa7700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f98880cdc90 0x7f988815f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.84:43868/0 (socket says 172.21.15.84:43868) 2022-12-01T09:47:41.770 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988dfa9700 1 -- 172.21.15.84:0/690971782 learned_addr learned my addr 172.21.15.84:0/690971782 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:41.771 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988dfa9700 1 -- 172.21.15.84:0/690971782 >> v2:172.21.15.191:3300/0 conn(0x7f988815b950 msgr2=0x7f98880cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:41.771 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988dfa9700 1 --2- 172.21.15.84:0/690971782 >> v2:172.21.15.191:3300/0 conn(0x7f988815b950 0x7f98880cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.771 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988dfa9700 1 -- 172.21.15.84:0/690971782 >> v1:172.21.15.191:6789/0 conn(0x7f98880cdc90 legacy=0x7f988815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:41.771 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988dfa9700 1 -- 172.21.15.84:0/690971782 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988806cf20 con 0x7f988815fc10 2022-12-01T09:47:41.772 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988d7a8700 1 --2- 172.21.15.84:0/690971782 >> v2:172.21.15.191:3300/0 conn(0x7f988815b950 0x7f98880cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:47:41.772 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f988dfa9700 1 --2- 172.21.15.84:0/690971782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f988815fc10 0x7f98880cc0e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9880009e50 tx=0x7f988000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be89e707a53b913 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:41.772 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f9877fff700 1 -- 172.21.15.84:0/690971782 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000e070 con 0x7f988815fc10 2022-12-01T09:47:41.773 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f9877fff700 1 -- 172.21.15.84:0/690971782 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988000b910 con 0x7f988815fc10 2022-12-01T09:47:41.773 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.762+0000 7f9877fff700 1 -- 172.21.15.84:0/690971782 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000bcc0 con 0x7f988815fc10 2022-12-01T09:47:41.773 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.763+0000 7f988fa0c700 1 -- 172.21.15.84:0/690971782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f988815fc10 msgr2=0x7f98880cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:41.774 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.763+0000 7f988fa0c700 1 --2- 172.21.15.84:0/690971782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f988815fc10 0x7f98880cc0e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9880009e50 tx=0x7f988000b330 comp rx=0 tx=0).stop 2022-12-01T09:47:41.774 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.763+0000 7f988fa0c700 1 -- 172.21.15.84:0/690971782 shutdown_connections 2022-12-01T09:47:41.774 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.763+0000 7f988fa0c700 1 --2- 172.21.15.84:0/690971782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f988815fc10 0x7f98880cc0e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.774 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.763+0000 7f988fa0c700 1 --2- 172.21.15.84:0/690971782 >> v2:172.21.15.191:3300/0 conn(0x7f988815b950 0x7f98880cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.774 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.763+0000 7f988fa0c700 1 -- 172.21.15.84:0/690971782 >> 172.21.15.84:0/690971782 conn(0x7f988806c260 msgr2=0x7f98880caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:41.775 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.763+0000 7f988fa0c700 1 -- 172.21.15.84:0/690971782 shutdown_connections 2022-12-01T09:47:41.775 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.763+0000 7f988fa0c700 1 -- 172.21.15.84:0/690971782 wait complete. 2022-12-01T09:47:41.778 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.764+0000 7f988fa0c700 1 Processor -- start 2022-12-01T09:47:41.778 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.764+0000 7f988fa0c700 1 -- start start 2022-12-01T09:47:41.779 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988fa0c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98880cdc90 0x7f98881f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:41.779 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988fa0c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988815b950 0x7f98881f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:41.779 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988fa0c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f988815fc10 0x7f98881f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:41.779 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988fa0c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98881f56e0 con 0x7f988815b950 2022-12-01T09:47:41.779 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988fa0c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98881f1740 con 0x7f988815fc10 2022-12-01T09:47:41.780 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988fa0c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98881f5120 con 0x7f98880cdc90 2022-12-01T09:47:41.780 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988cfa7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988815b950 0x7f98881f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:41.780 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988cfa7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988815b950 0x7f98881f1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47732/0 (socket says 172.21.15.84:47732) 2022-12-01T09:47:41.780 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988cfa7700 1 -- 172.21.15.84:0/137452903 learned_addr learned my addr 172.21.15.84:0/137452903 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:41.781 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988d7a8700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98880cdc90 0x7f98881f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:41.781 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988dfa9700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f988815fc10 0x7f98881f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:41.781 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988cfa7700 1 -- 172.21.15.84:0/137452903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98880cdc90 msgr2=0x7f98881f0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:41.781 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988cfa7700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98880cdc90 0x7f98881f0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.782 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988cfa7700 1 -- 172.21.15.84:0/137452903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f988815fc10 msgr2=0x7f98881f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:41.782 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.765+0000 7f988cfa7700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f988815fc10 0x7f98881f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.782 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.766+0000 7f988cfa7700 1 -- 172.21.15.84:0/137452903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780094a0 con 0x7f988815b950 2022-12-01T09:47:41.782 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.766+0000 7f988cfa7700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988815b950 0x7f98881f1b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f988400c520 tx=0x7f988400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:41.783 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.766+0000 7f9875ffb700 1 -- 172.21.15.84:0/137452903 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884005cb0 con 0x7f988815b950 2022-12-01T09:47:41.783 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.766+0000 7f9875ffb700 1 -- 172.21.15.84:0/137452903 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9884005e10 con 0x7f988815b950 2022-12-01T09:47:41.783 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.766+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9880009b30 con 0x7f988815b950 2022-12-01T09:47:41.783 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.766+0000 7f9875ffb700 1 -- 172.21.15.84:0/137452903 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98840072d0 con 0x7f988815b950 2022-12-01T09:47:41.784 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.766+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98881f1360 con 0x7f988815b950 2022-12-01T09:47:41.784 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.767+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9888067420 con 0x7f988815b950 2022-12-01T09:47:41.784 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.767+0000 7f9875ffb700 1 -- 172.21.15.84:0/137452903 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f9884019030 con 0x7f988815b950 2022-12-01T09:47:41.784 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.768+0000 7f9875ffb700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9868066590 0x7f9868068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:41.785 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.768+0000 7f988d7a8700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9868066590 0x7f9868068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:41.785 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.768+0000 7f9875ffb700 1 -- 172.21.15.84:0/137452903 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9884082430 con 0x7f988815b950 2022-12-01T09:47:41.785 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.768+0000 7f988d7a8700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9868066590 0x7f9868068a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9878009d70 tx=0x7f9878009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:41.785 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.772+0000 7f9875ffb700 1 -- 172.21.15.84:0/137452903 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f988404abe0 con 0x7f988815b950 2022-12-01T09:47:41.955 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.951+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f988806d200 con 0x7f988815b950 2022-12-01T09:47:41.956 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.952+0000 7f9875ffb700 1 -- 172.21.15.84:0/137452903 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12221 (secure 0 0 0) 0x7f9884017090 con 0x7f988815b950 2022-12-01T09:47:41.956 INFO:teuthology.misc.health.smithi084.stdout: 2022-12-01T09:47:41.957 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":16,"fsid":"8329cc4d-9cf5-499b-b4c9-a9793681da24","created":"2022-12-01T09:47:23.909978+0000","modified":"2022-12-01T09:47:38.419803+0000","last_up_change":"2022-12-01T09:47:35.194323+0000","last_in_change":"2022-12-01T09:47:26.403178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-12-01T09:47:35.244202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"e9c852ba-df6e-4c09-98f8-692b089e63c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6801","nonce":101363},{"type":"v1","addr":"172.21.15.84:6803","nonce":101363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6805","nonce":101363},{"type":"v1","addr":"172.21.15.84:6807","nonce":101363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6813","nonce":101363},{"type":"v1","addr":"172.21.15.84:6815","nonce":101363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":101363},{"type":"v1","addr":"172.21.15.84:6811","nonce":101363}]},"public_addr":"172.21.15.84:6803/101363","cluster_addr":"172.21.15.84:6807/101363","heartbeat_back_addr":"172.21.15.84:6815/101363","heartbeat_front_addr":"172.21.15.84:6811/101363","state":["exists","up"]},{"osd":1,"uuid":"2b47da52-0666-49d8-b263-725119e42db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":101364},{"type":"v1","addr":"172.21.15.84:6825","nonce":101364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":101364},{"type":"v1","addr":"172.21.15.84:6827","nonce":101364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":101364},{"type":"v1","addr":"172.21.15.84:6831","nonce":101364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":101364},{"type":"v1","addr":"172.21.15.84:6829","nonce":101364}]},"public_addr":"172.21.15.84:6825/101364","cluster_addr":"172.21.15.84:6827/101364","heartbeat_back_addr":"172.21.15.84:6831/101364","heartbeat_front_addr":"172.21.15.84:6829/101364","state":["exists","up"]},{"osd":2,"uuid":"7b668a53-55d9-481c-83a5-df590cd91fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":101362},{"type":"v1","addr":"172.21.15.84:6802","nonce":101362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":101362},{"type":"v1","addr":"172.21.15.84:6806","nonce":101362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":101362},{"type":"v1","addr":"172.21.15.84:6814","nonce":101362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":101362},{"type":"v1","addr":"172.21.15.84:6810","nonce":101362}]},"public_addr":"172.21.15.84:6802/101362","cluster_addr":"172.21.15.84:6806/101362","heartbeat_back_addr":"172.21.15.84:6814/101362","heartbeat_front_addr":"172.21.15.84:6810/101362","state":["exists","up"]},{"osd":3,"uuid":"1f2c87e3-7398-4f50-a56c-5f798d2fb279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":101365},{"type":"v1","addr":"172.21.15.84:6817","nonce":101365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":101365},{"type":"v1","addr":"172.21.15.84:6819","nonce":101365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":101365},{"type":"v1","addr":"172.21.15.84:6823","nonce":101365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":101365},{"type":"v1","addr":"172.21.15.84:6821","nonce":101365}]},"public_addr":"172.21.15.84:6817/101365","cluster_addr":"172.21.15.84:6819/101365","heartbeat_back_addr":"172.21.15.84:6823/101365","heartbeat_front_addr":"172.21.15.84:6821/101365","state":["exists","up"]},{"osd":4,"uuid":"8a4b43a4-6c63-48ab-826a-cb8919c44b8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":101621},{"type":"v1","addr":"172.21.15.191:6801","nonce":101621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":101621},{"type":"v1","addr":"172.21.15.191:6803","nonce":101621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":101621},{"type":"v1","addr":"172.21.15.191:6807","nonce":101621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":101621},{"type":"v1","addr":"172.21.15.191:6805","nonce":101621}]},"public_addr":"172.21.15.191:6801/101621","cluster_addr":"172.21.15.191:6803/101621","heartbeat_back_addr":"172.21.15.191:6807/101621","heartbeat_front_addr":"172.21.15.191:6805/101621","state":["exists","up"]},{"osd":5,"uuid":"18a3160f-c761-49e7-8d7d-56fdf4c8a7f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":101635},{"type":"v1","addr":"172.21.15.191:6817","nonce":101635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":101635},{"type":"v1","addr":"172.21.15.191:6819","nonce":101635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":101635},{"type":"v1","addr":"172.21.15.191:6823","nonce":101635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":101635},{"type":"v1","addr":"172.21.15.191:6821","nonce":101635}]},"public_addr":"172.21.15.191:6817/101635","cluster_addr":"172.21.15.191:6819/101635","heartbeat_back_addr":"172.21.15.191:6823/101635","heartbeat_front_addr":"172.21.15.191:6821/101635","state":["exists","up"]},{"osd":6,"uuid":"e70b8522-3456-479c-81c9-10e06ad3172f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":101622},{"type":"v1","addr":"172.21.15.191:6809","nonce":101622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":101622},{"type":"v1","addr":"172.21.15.191:6811","nonce":101622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":101622},{"type":"v1","addr":"172.21.15.191:6815","nonce":101622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":101622},{"type":"v1","addr":"172.21.15.191:6813","nonce":101622}]},"public_addr":"172.21.15.191:6809/101622","cluster_addr":"172.21.15.191:6811/101622","heartbeat_back_addr":"172.21.15.191:6815/101622","heartbeat_front_addr":"172.21.15.191:6813/101622","state":["exists","up"]},{"osd":7,"uuid":"d2c33094-6303-4ed3-963c-e010df6aa820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":101646},{"type":"v1","addr":"172.21.15.191:6825","nonce":101646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":101646},{"type":"v1","addr":"172.21.15.191:6827","nonce":101646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":101646},{"type":"v1","addr":"172.21.15.191:6831","nonce":101646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":101646},{"type":"v1","addr":"172.21.15.191:6829","nonce":101646}]},"public_addr":"172.21.15.191:6825/101646","cluster_addr":"172.21.15.191:6827/101646","heartbeat_back_addr":"172.21.15.191:6831/101646","heartbeat_front_addr":"172.21.15.191:6829/101646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.585643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.563418+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.596853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.531044+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.550829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.604675+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.514679+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.557085+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:47:41.959 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9868066590 msgr2=0x7f9868068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:41.959 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9868066590 0x7f9868068a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9878009d70 tx=0x7f9878009210 comp rx=0 tx=0).stop 2022-12-01T09:47:41.959 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988815b950 msgr2=0x7f98881f1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:41.959 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988815b950 0x7f98881f1b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f988400c520 tx=0x7f988400c830 comp rx=0 tx=0).stop 2022-12-01T09:47:41.960 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 shutdown_connections 2022-12-01T09:47:41.960 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f988815fc10 0x7f98881f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.960 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98880cdc90 0x7f98881f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.960 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9868066590 0x7f9868068a40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.960 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 --2- 172.21.15.84:0/137452903 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f988815b950 0x7f98881f1b60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:41.961 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 >> 172.21.15.84:0/137452903 conn(0x7f988806c260 msgr2=0x7f988806fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:41.961 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 shutdown_connections 2022-12-01T09:47:41.962 INFO:teuthology.misc.health.smithi084.stderr:2022-12-01T09:47:41.954+0000 7f988fa0c700 1 -- 172.21.15.84:0/137452903 wait complete. 2022-12-01T09:47:41.969 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-12-01T09:47:41.970 INFO:tasks.ceph:Creating RBD pool 2022-12-01T09:47:41.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd pool create rbd 8 2022-12-01T09:47:42.088 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.084+0000 7f86459b2700 1 Processor -- start 2022-12-01T09:47:42.089 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.084+0000 7f86459b2700 1 -- start start 2022-12-01T09:47:42.092 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f86459b2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8640159710 0x7f86400b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.092 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f86459b2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f86400b5b60 0x7f864006d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.092 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f86459b2700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f864006d540 0x7f86400b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.093 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f86459b2700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864006d9c0 con 0x7f864006d540 2022-12-01T09:47:42.093 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f86459b2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86400cc7a0 con 0x7f86400b5b60 2022-12-01T09:47:42.093 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f86459b2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86400b5930 con 0x7f8640159710 2022-12-01T09:47:42.093 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863f7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f864006d540 0x7f86400b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.094 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863f7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f864006d540 0x7f86400b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47742/0 (socket says 172.21.15.84:47742) 2022-12-01T09:47:42.094 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863f7fe700 1 -- 172.21.15.84:0/2978789165 learned_addr learned my addr 172.21.15.84:0/2978789165 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:42.094 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863e7fc700 1 --2- 172.21.15.84:0/2978789165 >> v2:172.21.15.191:3300/0 conn(0x7f86400b5b60 0x7f864006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.094 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863effd700 1 --2- 172.21.15.84:0/2978789165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8640159710 0x7f86400b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.094 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863f7fe700 1 -- 172.21.15.84:0/2978789165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8640159710 msgr2=0x7f86400b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:42.095 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863f7fe700 1 --2- 172.21.15.84:0/2978789165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8640159710 0x7f86400b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.095 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863f7fe700 1 -- 172.21.15.84:0/2978789165 >> v2:172.21.15.191:3300/0 conn(0x7f86400b5b60 msgr2=0x7f864006d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:42.095 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863f7fe700 1 --2- 172.21.15.84:0/2978789165 >> v2:172.21.15.191:3300/0 conn(0x7f86400b5b60 0x7f864006d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.095 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.085+0000 7f863f7fe700 1 -- 172.21.15.84:0/2978789165 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400cc1e0 con 0x7f864006d540 2022-12-01T09:47:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f863f7fe700 1 --2- 172.21.15.84:0/2978789165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f864006d540 0x7f86400b19b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f86400b2ab0 tx=0x7f863800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d50eae76198bd24 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f863dffb700 1 -- 172.21.15.84:0/2978789165 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863800e070 con 0x7f864006d540 2022-12-01T09:47:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f863dffb700 1 -- 172.21.15.84:0/2978789165 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863800b9c0 con 0x7f864006d540 2022-12-01T09:47:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f863dffb700 1 -- 172.21.15.84:0/2978789165 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863800bd70 con 0x7f864006d540 2022-12-01T09:47:42.097 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 -- 172.21.15.84:0/2978789165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f864006d540 msgr2=0x7f86400b19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.097 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 --2- 172.21.15.84:0/2978789165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f864006d540 0x7f86400b19b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f86400b2ab0 tx=0x7f863800b330 comp rx=0 tx=0).stop 2022-12-01T09:47:42.097 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 -- 172.21.15.84:0/2978789165 shutdown_connections 2022-12-01T09:47:42.097 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 --2- 172.21.15.84:0/2978789165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f864006d540 0x7f86400b19b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.097 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 --2- 172.21.15.84:0/2978789165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8640159710 0x7f86400b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 --2- 172.21.15.84:0/2978789165 >> v2:172.21.15.191:3300/0 conn(0x7f86400b5b60 0x7f864006d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 -- 172.21.15.84:0/2978789165 >> 172.21.15.84:0/2978789165 conn(0x7f86400cff30 msgr2=0x7f86400d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 -- 172.21.15.84:0/2978789165 shutdown_connections 2022-12-01T09:47:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.086+0000 7f86459b2700 1 -- 172.21.15.84:0/2978789165 wait complete. 2022-12-01T09:47:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.087+0000 7f86459b2700 1 Processor -- start 2022-12-01T09:47:42.099 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.088+0000 7f86459b2700 1 -- start start 2022-12-01T09:47:42.102 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.088+0000 7f86459b2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f864006d540 0x7f86401537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.102 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.088+0000 7f86459b2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400b5b60 0x7f86400b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.102 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.088+0000 7f86459b2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8640159710 0x7f86400b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.102 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.088+0000 7f86459b2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400c5340 con 0x7f86400b5b60 2022-12-01T09:47:42.103 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.088+0000 7f86459b2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401ef010 con 0x7f8640159710 2022-12-01T09:47:42.103 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.088+0000 7f86459b2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86400c47c0 con 0x7f864006d540 2022-12-01T09:47:42.103 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.088+0000 7f863e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400b5b60 0x7f86400b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.103 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400b5b60 0x7f86400b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47746/0 (socket says 172.21.15.84:47746) 2022-12-01T09:47:42.104 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863e7fc700 1 -- 172.21.15.84:0/1034360721 learned_addr learned my addr 172.21.15.84:0/1034360721 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:42.104 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863f7fe700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8640159710 0x7f86400b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.104 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863effd700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f864006d540 0x7f86401537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.104 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863e7fc700 1 -- 172.21.15.84:0/1034360721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f864006d540 msgr2=0x7f86401537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.104 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863e7fc700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f864006d540 0x7f86401537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.105 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863e7fc700 1 -- 172.21.15.84:0/1034360721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8640159710 msgr2=0x7f86400b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.105 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863e7fc700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8640159710 0x7f86400b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.105 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863e7fc700 1 -- 172.21.15.84:0/1034360721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8638009b30 con 0x7f86400b5b60 2022-12-01T09:47:42.105 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f863e7fc700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400b5b60 0x7f86400b36e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f863000c820 tx=0x7f863000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:42.105 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f8627fff700 1 -- 172.21.15.84:0/1034360721 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630011800 con 0x7f86400b5b60 2022-12-01T09:47:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f8627fff700 1 -- 172.21.15.84:0/1034360721 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630011960 con 0x7f86400b5b60 2022-12-01T09:47:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f8627fff700 1 -- 172.21.15.84:0/1034360721 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630011d10 con 0x7f86400b5b60 2022-12-01T09:47:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86400c4aa0 con 0x7f86400b5b60 2022-12-01T09:47:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.089+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86400c5060 con 0x7f86400b5b60 2022-12-01T09:47:42.106 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.090+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86401ef1a0 con 0x7f86400b5b60 2022-12-01T09:47:42.107 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.090+0000 7f8627fff700 1 -- 172.21.15.84:0/1034360721 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f8630021070 con 0x7f86400b5b60 2022-12-01T09:47:42.107 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.091+0000 7f8627fff700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f8620066590 0x7f8620068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.107 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.091+0000 7f8627fff700 1 -- 172.21.15.84:0/1034360721 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f86300830b0 con 0x7f86400b5b60 2022-12-01T09:47:42.108 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.091+0000 7f863effd700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f8620066590 0x7f8620068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.108 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.091+0000 7f863effd700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f8620066590 0x7f8620068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8634008850 tx=0x7f8634008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:42.109 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.095+0000 7f8627fff700 1 -- 172.21.15.84:0/1034360721 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f8630050c70 con 0x7f86400b5b60 2022-12-01T09:47:42.278 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.274+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f8640067420 con 0x7f86400b5b60 2022-12-01T09:47:42.535 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-12-01T09:47:42.538 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-12-01T09:47:42.678 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.674+0000 7f8627fff700 1 -- 172.21.15.84:0/1034360721 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f863001f0a0 con 0x7f86400b5b60 2022-12-01T09:47:42.680 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f8620066590 msgr2=0x7f8620068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.680 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f8620066590 0x7f8620068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8634008850 tx=0x7f8634008040 comp rx=0 tx=0).stop 2022-12-01T09:47:42.681 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400b5b60 msgr2=0x7f86400b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.681 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400b5b60 0x7f86400b36e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f863000c820 tx=0x7f863000cbe0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.681 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 shutdown_connections 2022-12-01T09:47:42.681 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8640159710 0x7f86400b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.682 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f864006d540 0x7f86401537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.682 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f8620066590 0x7f8620068a40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.682 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 --2- 172.21.15.84:0/1034360721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86400b5b60 0x7f86400b36e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.683 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 >> 172.21.15.84:0/1034360721 conn(0x7f86400cff30 msgr2=0x7f86400b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:42.683 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 shutdown_connections 2022-12-01T09:47:42.684 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.676+0000 7f86459b2700 1 -- 172.21.15.84:0/1034360721 wait complete. 2022-12-01T09:47:42.685 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'rbd' created 2022-12-01T09:47:42.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-12-01T09:47:42.815 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.810+0000 7fc0aabd1700 1 Processor -- start 2022-12-01T09:47:42.815 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.810+0000 7fc0aabd1700 1 -- start start 2022-12-01T09:47:42.821 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.811+0000 7fc0aabd1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a4155fd0 0x7fc0a4071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.822 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.811+0000 7fc0aabd1700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a406dd10 con 0x7fc0a40cda70 2022-12-01T09:47:42.822 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.811+0000 7fc0aabd1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a415bf70 con 0x7fc0a406d8f0 2022-12-01T09:47:42.822 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.811+0000 7fc0aabd1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a415cbf0 con 0x7fc0a4155fd0 2022-12-01T09:47:42.822 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.811+0000 7fc0a3fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc0a40cda70 0x7fc0a406d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52602/0 (socket says 172.21.15.84:52602) 2022-12-01T09:47:42.823 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.811+0000 7fc0a3fff700 1 -- 172.21.15.84:0/3383198899 learned_addr learned my addr 172.21.15.84:0/3383198899 (peer_addr_for_me v1:172.21.15.84:0/0) 2022-12-01T09:47:42.823 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a896d700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a4155fd0 0x7fc0a4071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.823 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801399802 0 0) 0x7fc0a406dd10 con 0x7fc0a40cda70 2022-12-01T09:47:42.823 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0880035b0 con 0x7fc0a40cda70 2022-12-01T09:47:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1558904421 0 0) 0x7fc0880035b0 con 0x7fc0a40cda70 2022-12-01T09:47:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a4155fd0 msgr2=0x7fc0a4071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a4155fd0 0x7fc0a4071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.824 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 >> v1:172.21.15.191:6789/0 conn(0x7fc0a406d8f0 legacy=0x7fc0a415f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a415c250 con 0x7fc0a40cda70 2022-12-01T09:47:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2125368202 0 0) 0x7fc090001670 con 0x7fc0a40cda70 2022-12-01T09:47:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 >> v1:172.21.15.84:6789/0 conn(0x7fc0a40cda70 legacy=0x7fc0a406d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.825 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a4155fd0 0x7fc088003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc088003f70 0x7fc088004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc088004d90 0x7fc088005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a415cbf0 con 0x7fc088004d90 2022-12-01T09:47:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08800baf0 con 0x7fc088003f70 2022-12-01T09:47:42.826 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08800bcb0 con 0x7fc0a4155fd0 2022-12-01T09:47:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a916e700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc088004d90 0x7fc088005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a896d700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a4155fd0 0x7fc088003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a3fff700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc088003f70 0x7fc088004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.827 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a916e700 1 -- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a4155fd0 msgr2=0x7fc088003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:42.828 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a916e700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a4155fd0 0x7fc088003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.828 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a916e700 1 -- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc088003f70 msgr2=0x7fc088004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:42.828 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a916e700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc088003f70 0x7fc088004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.828 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.812+0000 7fc0a916e700 1 -- 172.21.15.84:0/3383198899 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0880035b0 con 0x7fc088004d90 2022-12-01T09:47:42.828 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0a916e700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc088004d90 0x7fc088005200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c340 tx=0x7fc088005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a800b6beedbb49bb server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:42.829 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c001f20 con 0x7fc088004d90 2022-12-01T09:47:42.829 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c00ce80 con 0x7fc088004d90 2022-12-01T09:47:42.829 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0a37fe700 1 -- 172.21.15.84:0/3383198899 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c012660 con 0x7fc088004d90 2022-12-01T09:47:42.829 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3383198899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc088004d90 msgr2=0x7fc088005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.830 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0aabd1700 1 --2- 172.21.15.84:0/3383198899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc088004d90 0x7fc088005200 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c340 tx=0x7fc088005880 comp rx=0 tx=0).stop 2022-12-01T09:47:42.830 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0a916e700 1 -- 172.21.15.84:0/3383198899 reap_dead start 2022-12-01T09:47:42.830 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3383198899 shutdown_connections 2022-12-01T09:47:42.830 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3383198899 >> 172.21.15.84:0/3383198899 conn(0x7fc0a40ced60 msgr2=0x7fc0a40cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:42.831 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3383198899 shutdown_connections 2022-12-01T09:47:42.831 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.813+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3383198899 wait complete. 2022-12-01T09:47:42.831 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.814+0000 7fc0aabd1700 1 Processor -- start 2022-12-01T09:47:42.832 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.814+0000 7fc0aabd1700 1 -- start start 2022-12-01T09:47:42.832 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0aabd1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a40b4330 0x7fc0a40b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.833 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0aabd1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a40b32a0 0x7fc0a415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.833 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0aabd1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0a40b4700 0x7fc0a40c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.834 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0aabd1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40b2200 con 0x7fc0a40b4330 2022-12-01T09:47:42.834 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0aabd1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40b24e0 con 0x7fc0a40b4700 2022-12-01T09:47:42.835 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0aabd1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a40b27c0 con 0x7fc0a40b32a0 2022-12-01T09:47:42.835 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0a896d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a40b4330 0x7fc0a40b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.836 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0a896d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a40b4330 0x7fc0a40b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47764/0 (socket says 172.21.15.84:47764) 2022-12-01T09:47:42.836 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.815+0000 7fc0a896d700 1 -- 172.21.15.84:0/3473696615 learned_addr learned my addr 172.21.15.84:0/3473696615 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:42.836 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a3fff700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a40b32a0 0x7fc0a415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.837 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a916e700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0a40b4700 0x7fc0a40c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.837 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a896d700 1 -- 172.21.15.84:0/3473696615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a40b32a0 msgr2=0x7fc0a415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.838 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a896d700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a40b32a0 0x7fc0a415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.838 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a896d700 1 -- 172.21.15.84:0/3473696615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0a40b4700 msgr2=0x7fc0a40c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:42.838 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a896d700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0a40b4700 0x7fc0a40c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:42.839 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a896d700 1 -- 172.21.15.84:0/3473696615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a40b2aa0 con 0x7fc0a40b4330 2022-12-01T09:47:42.839 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a896d700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a40b4330 0x7fc0a40b1b90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc098011aa0 tx=0x7fc098011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:42.840 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a17fa700 1 -- 172.21.15.84:0/3473696615 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800f980 con 0x7fc0a40b4330 2022-12-01T09:47:42.840 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a17fa700 1 -- 172.21.15.84:0/3473696615 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09800fae0 con 0x7fc0a40b4330 2022-12-01T09:47:42.840 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a40b2d80 con 0x7fc0a40b4330 2022-12-01T09:47:42.841 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0a17fa700 1 -- 172.21.15.84:0/3473696615 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800fe90 con 0x7fc0a40b4330 2022-12-01T09:47:42.841 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.816+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a40c4c00 con 0x7fc0a40b4330 2022-12-01T09:47:42.842 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.817+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a40b2fa0 con 0x7fc0a40b4330 2022-12-01T09:47:42.842 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.818+0000 7fc0a17fa700 1 -- 172.21.15.84:0/3473696615 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fc098014440 con 0x7fc0a40b4330 2022-12-01T09:47:42.843 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.818+0000 7fc0a17fa700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fc08c066590 0x7fc08c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:42.843 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.818+0000 7fc0a3fff700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fc08c066590 0x7fc08c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:42.844 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.818+0000 7fc0a17fa700 1 -- 172.21.15.84:0/3473696615 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fc098083dd0 con 0x7fc0a40b4330 2022-12-01T09:47:42.844 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.818+0000 7fc0a3fff700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fc08c066590 0x7fc08c068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40b3b90 tx=0x7fc090009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:42.844 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:42.822+0000 7fc0a17fa700 1 -- 172.21.15.84:0/3473696615 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7fc0980518a0 con 0x7fc0a40b4330 2022-12-01T09:47:43.006 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.001+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc0a4067420 con 0x7fc0a40b4330 2022-12-01T09:47:43.693 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.688+0000 7fc0a17fa700 1 -- 172.21.15.84:0/3473696615 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc098022020 con 0x7fc0a40b4330 2022-12-01T09:47:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fc08c066590 msgr2=0x7fc08c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fc08c066590 0x7fc08c068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40b3b90 tx=0x7fc090009040 comp rx=0 tx=0).stop 2022-12-01T09:47:43.696 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a40b4330 msgr2=0x7fc0a40b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a40b4330 0x7fc0a40b1b90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc098011aa0 tx=0x7fc098011e60 comp rx=0 tx=0).stop 2022-12-01T09:47:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 shutdown_connections 2022-12-01T09:47:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0a40b4700 0x7fc0a40c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.697 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fc08c066590 0x7fc08c068a40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.698 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0a40b4330 0x7fc0a40b1b90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.698 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 --2- 172.21.15.84:0/3473696615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc0a40b32a0 0x7fc0a415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.698 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 >> 172.21.15.84:0/3473696615 conn(0x7fc0a40ced60 msgr2=0x7fc0a40aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:43.698 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 shutdown_connections 2022-12-01T09:47:43.698 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.691+0000 7fc0aabd1700 1 -- 172.21.15.84:0/3473696615 wait complete. 2022-12-01T09:47:43.699 INFO:teuthology.orchestra.run.smithi084.stderr:enabled application 'rbd' on pool 'rbd' 2022-12-01T09:47:43.709 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-12-01T09:47:43.710 INFO:tasks.ceph.mds.a:Restarting daemon 2022-12-01T09:47:43.710 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-12-01T09:47:43.713 INFO:tasks.ceph.mds.a:Started 2022-12-01T09:47:43.713 INFO:tasks.ceph.mds.c:Restarting daemon 2022-12-01T09:47:43.714 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-12-01T09:47:43.716 INFO:tasks.ceph.mds.c:Started 2022-12-01T09:47:43.716 INFO:tasks.ceph.mds.b:Restarting daemon 2022-12-01T09:47:43.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-12-01T09:47:43.719 INFO:tasks.ceph.mds.b:Started 2022-12-01T09:47:43.720 INFO:tasks.ceph.mds.d:Restarting daemon 2022-12-01T09:47:43.720 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-12-01T09:47:43.722 INFO:tasks.ceph.mds.d:Started 2022-12-01T09:47:43.723 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-12-01T09:47:43.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-12-01T09:47:43.793 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2022-12-01T09:47:43.794 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2022-12-01T09:47:43.806 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2022-12-01T09:47:43.807 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2022-12-01T09:47:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.848+0000 7faa8a175700 1 Processor -- start 2022-12-01T09:47:43.853 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa8a175700 1 -- start start 2022-12-01T09:47:43.856 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa8a175700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa8415b9d0 0x7faa8415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:43.856 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa8a175700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faa840cc940 0x7faa84071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:43.857 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa8a175700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8415f7f0 con 0x7faa840ae3f0 2022-12-01T09:47:43.857 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa8a175700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8415bda0 con 0x7faa840cc940 2022-12-01T09:47:43.857 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa8a175700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa840cceb0 con 0x7faa8415b9d0 2022-12-01T09:47:43.858 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa82ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faa840cc940 0x7faa84071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:43.858 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa82ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faa840cc940 0x7faa84071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47800/0 (socket says 172.21.15.84:47800) 2022-12-01T09:47:43.858 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.849+0000 7faa82ffd700 1 -- 172.21.15.84:0/3633150541 learned_addr learned my addr 172.21.15.84:0/3633150541 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa837fe700 1 --2- 172.21.15.84:0/3633150541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa8415b9d0 0x7faa8415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa82ffd700 1 -- 172.21.15.84:0/3633150541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa8415b9d0 msgr2=0x7faa8415fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa82ffd700 1 --2- 172.21.15.84:0/3633150541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa8415b9d0 0x7faa8415fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.859 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa82ffd700 1 -- 172.21.15.84:0/3633150541 >> v1:172.21.15.191:6789/0 conn(0x7faa840ae3f0 legacy=0x7faa84069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa82ffd700 1 -- 172.21.15.84:0/3633150541 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa840ce2f0 con 0x7faa840cc940 2022-12-01T09:47:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa82ffd700 1 --2- 172.21.15.84:0/3633150541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa840cc940 0x7faa84071f60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa780098f0 tx=0x7faa78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cef95fb572ae73 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa827fc700 1 -- 172.21.15.84:0/3633150541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78004230 con 0x7faa840cc940 2022-12-01T09:47:43.860 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa827fc700 1 -- 172.21.15.84:0/3633150541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa78004390 con 0x7faa840cc940 2022-12-01T09:47:43.861 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.850+0000 7faa827fc700 1 -- 172.21.15.84:0/3633150541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa780036a0 con 0x7faa840cc940 2022-12-01T09:47:43.861 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.851+0000 7faa8a175700 1 -- 172.21.15.84:0/3633150541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa840cc940 msgr2=0x7faa84071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:43.861 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.851+0000 7faa8a175700 1 --2- 172.21.15.84:0/3633150541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa840cc940 0x7faa84071f60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa780098f0 tx=0x7faa78009c00 comp rx=0 tx=0).stop 2022-12-01T09:47:43.861 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.851+0000 7faa8a175700 1 -- 172.21.15.84:0/3633150541 shutdown_connections 2022-12-01T09:47:43.862 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.851+0000 7faa8a175700 1 --2- 172.21.15.84:0/3633150541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa8415b9d0 0x7faa8415fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.862 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.851+0000 7faa8a175700 1 --2- 172.21.15.84:0/3633150541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa840cc940 0x7faa84071f60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.862 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.851+0000 7faa8a175700 1 -- 172.21.15.84:0/3633150541 >> 172.21.15.84:0/3633150541 conn(0x7faa840cb7f0 msgr2=0x7faa840cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:43.862 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.851+0000 7faa8a175700 1 -- 172.21.15.84:0/3633150541 shutdown_connections 2022-12-01T09:47:43.862 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.851+0000 7faa8a175700 1 -- 172.21.15.84:0/3633150541 wait complete. 2022-12-01T09:47:43.863 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.852+0000 7faa8a175700 1 Processor -- start 2022-12-01T09:47:43.863 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.852+0000 7faa8a175700 1 -- start start 2022-12-01T09:47:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa8a175700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa840ae3f0 0x7faa84151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa8a175700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa840cc940 0x7faa841f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa8a175700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa8415b9d0 0x7faa840ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:43.866 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa8a175700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa84151c40 con 0x7faa8415b9d0 2022-12-01T09:47:43.867 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa8a175700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa841f5c20 con 0x7faa840ae3f0 2022-12-01T09:47:43.867 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa8a175700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa84158ac0 con 0x7faa840cc940 2022-12-01T09:47:43.867 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa837fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa840ae3f0 0x7faa84151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:43.867 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa83fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa8415b9d0 0x7faa840ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:43.868 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa82ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa840cc940 0x7faa841f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:43.868 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa837fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa840ae3f0 0x7faa84151700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.84:58100/0 (socket says 172.21.15.84:58100) 2022-12-01T09:47:43.868 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa837fe700 1 -- 172.21.15.84:0/1201111350 learned_addr learned my addr 172.21.15.84:0/1201111350 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:43.869 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa83fff700 1 -- 172.21.15.84:0/1201111350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa840cc940 msgr2=0x7faa841f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:43.869 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa83fff700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa840cc940 0x7faa841f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.869 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa83fff700 1 -- 172.21.15.84:0/1201111350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa840ae3f0 msgr2=0x7faa84151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:43.870 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa83fff700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa840ae3f0 0x7faa84151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:43.870 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa83fff700 1 -- 172.21.15.84:0/1201111350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa78009550 con 0x7faa8415b9d0 2022-12-01T09:47:43.870 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa837fe700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa840ae3f0 0x7faa84151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:47:43.870 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa83fff700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa8415b9d0 0x7faa840ae950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00cb10 tx=0x7faa7c00ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:43.871 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa6bfff700 1 -- 172.21.15.84:0/1201111350 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00a9e0 con 0x7faa8415b9d0 2022-12-01T09:47:43.871 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa6bfff700 1 -- 172.21.15.84:0/1201111350 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c00ab40 con 0x7faa8415b9d0 2022-12-01T09:47:43.871 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa84158da0 con 0x7faa8415b9d0 2022-12-01T09:47:43.871 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa6bfff700 1 -- 172.21.15.84:0/1201111350 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c012420 con 0x7faa8415b9d0 2022-12-01T09:47:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.853+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa840b3bb0 con 0x7faa8415b9d0 2022-12-01T09:47:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.854+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa84067420 con 0x7faa8415b9d0 2022-12-01T09:47:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.855+0000 7faa6bfff700 1 -- 172.21.15.84:0/1201111350 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7faa7c022070 con 0x7faa8415b9d0 2022-12-01T09:47:43.872 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.855+0000 7faa6bfff700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7faa64066590 0x7faa64068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:43.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.855+0000 7faa6bfff700 1 -- 172.21.15.84:0/1201111350 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7faa7c083880 con 0x7faa8415b9d0 2022-12-01T09:47:43.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.855+0000 7faa837fe700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7faa64066590 0x7faa64068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:43.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.856+0000 7faa837fe700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7faa64066590 0x7faa64068a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faa74007ee0 tx=0x7faa74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:43.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:43.859+0000 7faa6bfff700 1 -- 172.21.15.84:0/1201111350 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7faa7c0512e0 con 0x7faa8415b9d0 2022-12-01T09:47:44.043 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.039+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa840ce570 con 0x7faa8415b9d0 2022-12-01T09:47:44.044 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.039+0000 7faa6bfff700 1 -- 172.21.15.84:0/1201111350 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13754 (secure 0 0 0) 0x7faa7c016090 con 0x7faa8415b9d0 2022-12-01T09:47:44.045 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:47:44.045 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"fsid":"8329cc4d-9cf5-499b-b4c9-a9793681da24","created":"2022-12-01T09:47:23.909978+0000","modified":"2022-12-01T09:47:43.678846+0000","last_up_change":"2022-12-01T09:47:35.194323+0000","last_in_change":"2022-12-01T09:47:26.403178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-12-01T09:47:35.244202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-12-01T09:47:42.293080+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9c852ba-df6e-4c09-98f8-692b089e63c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6801","nonce":101363},{"type":"v1","addr":"172.21.15.84:6803","nonce":101363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6805","nonce":101363},{"type":"v1","addr":"172.21.15.84:6807","nonce":101363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6813","nonce":101363},{"type":"v1","addr":"172.21.15.84:6815","nonce":101363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":101363},{"type":"v1","addr":"172.21.15.84:6811","nonce":101363}]},"public_addr":"172.21.15.84:6803/101363","cluster_addr":"172.21.15.84:6807/101363","heartbeat_back_addr":"172.21.15.84:6815/101363","heartbeat_front_addr":"172.21.15.84:6811/101363","state":["exists","up"]},{"osd":1,"uuid":"2b47da52-0666-49d8-b263-725119e42db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":101364},{"type":"v1","addr":"172.21.15.84:6825","nonce":101364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":101364},{"type":"v1","addr":"172.21.15.84:6827","nonce":101364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":101364},{"type":"v1","addr":"172.21.15.84:6831","nonce":101364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":101364},{"type":"v1","addr":"172.21.15.84:6829","nonce":101364}]},"public_addr":"172.21.15.84:6825/101364","cluster_addr":"172.21.15.84:6827/101364","heartbeat_back_addr":"172.21.15.84:6831/101364","heartbeat_front_addr":"172.21.15.84:6829/101364","state":["exists","up"]},{"osd":2,"uuid":"7b668a53-55d9-481c-83a5-df590cd91fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":101362},{"type":"v1","addr":"172.21.15.84:6802","nonce":101362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":101362},{"type":"v1","addr":"172.21.15.84:6806","nonce":101362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":101362},{"type":"v1","addr":"172.21.15.84:6814","nonce":101362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":101362},{"type":"v1","addr":"172.21.15.84:6810","nonce":101362}]},"public_addr":"172.21.15.84:6802/101362","cluster_addr":"172.21.15.84:6806/101362","heartbeat_back_addr":"172.21.15.84:6814/101362","heartbeat_front_addr":"172.21.15.84:6810/101362","state":["exists","up"]},{"osd":3,"uuid":"1f2c87e3-7398-4f50-a56c-5f798d2fb279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":101365},{"type":"v1","addr":"172.21.15.84:6817","nonce":101365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":101365},{"type":"v1","addr":"172.21.15.84:6819","nonce":101365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":101365},{"type":"v1","addr":"172.21.15.84:6823","nonce":101365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":101365},{"type":"v1","addr":"172.21.15.84:6821","nonce":101365}]},"public_addr":"172.21.15.84:6817/101365","cluster_addr":"172.21.15.84:6819/101365","heartbeat_back_addr":"172.21.15.84:6823/101365","heartbeat_front_addr":"172.21.15.84:6821/101365","state":["exists","up"]},{"osd":4,"uuid":"8a4b43a4-6c63-48ab-826a-cb8919c44b8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":101621},{"type":"v1","addr":"172.21.15.191:6801","nonce":101621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":101621},{"type":"v1","addr":"172.21.15.191:6803","nonce":101621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":101621},{"type":"v1","addr":"172.21.15.191:6807","nonce":101621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":101621},{"type":"v1","addr":"172.21.15.191:6805","nonce":101621}]},"public_addr":"172.21.15.191:6801/101621","cluster_addr":"172.21.15.191:6803/101621","heartbeat_back_addr":"172.21.15.191:6807/101621","heartbeat_front_addr":"172.21.15.191:6805/101621","state":["exists","up"]},{"osd":5,"uuid":"18a3160f-c761-49e7-8d7d-56fdf4c8a7f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":101635},{"type":"v1","addr":"172.21.15.191:6817","nonce":101635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":101635},{"type":"v1","addr":"172.21.15.191:6819","nonce":101635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":101635},{"type":"v1","addr":"172.21.15.191:6823","nonce":101635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":101635},{"type":"v1","addr":"172.21.15.191:6821","nonce":101635}]},"public_addr":"172.21.15.191:6817/101635","cluster_addr":"172.21.15.191:6819/101635","heartbeat_back_addr":"172.21.15.191:6823/101635","heartbeat_front_addr":"172.21.15.191:6821/101635","state":["exists","up"]},{"osd":6,"uuid":"e70b8522-3456-479c-81c9-10e06ad3172f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":101622},{"type":"v1","addr":"172.21.15.191:6809","nonce":101622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":101622},{"type":"v1","addr":"172.21.15.191:6811","nonce":101622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":101622},{"type":"v1","addr":"172.21.15.191:6815","nonce":101622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":101622},{"type":"v1","addr":"172.21.15.191:6813","nonce":101622}]},"public_addr":"172.21.15.191:6809/101622","cluster_addr":"172.21.15.191:6811/101622","heartbeat_back_addr":"172.21.15.191:6815/101622","heartbeat_front_addr":"172.21.15.191:6813/101622","state":["exists","up"]},{"osd":7,"uuid":"d2c33094-6303-4ed3-963c-e010df6aa820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":101646},{"type":"v1","addr":"172.21.15.191:6825","nonce":101646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":101646},{"type":"v1","addr":"172.21.15.191:6827","nonce":101646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":101646},{"type":"v1","addr":"172.21.15.191:6831","nonce":101646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":101646},{"type":"v1","addr":"172.21.15.191:6829","nonce":101646}]},"public_addr":"172.21.15.191:6825/101646","cluster_addr":"172.21.15.191:6827/101646","heartbeat_back_addr":"172.21.15.191:6831/101646","heartbeat_front_addr":"172.21.15.191:6829/101646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.585643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.563418+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.596853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.531044+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.550829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.604675+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.514679+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.557085+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:47:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7faa64066590 msgr2=0x7faa64068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7faa64066590 0x7faa64068a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faa74007ee0 tx=0x7faa74008040 comp rx=0 tx=0).stop 2022-12-01T09:47:44.049 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa8415b9d0 msgr2=0x7faa840ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.049 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa8415b9d0 0x7faa840ae950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00cb10 tx=0x7faa7c00ce20 comp rx=0 tx=0).stop 2022-12-01T09:47:44.050 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 shutdown_connections 2022-12-01T09:47:44.050 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7faa64066590 0x7faa64068a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.051 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa8415b9d0 0x7faa840ae950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.051 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa840ae3f0 0x7faa84151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.052 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 --2- 172.21.15.84:0/1201111350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa840cc940 0x7faa841f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.052 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 >> 172.21.15.84:0/1201111350 conn(0x7faa840cb7f0 msgr2=0x7faa84070850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:44.053 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 shutdown_connections 2022-12-01T09:47:44.053 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.041+0000 7faa8a175700 1 -- 172.21.15.84:0/1201111350 wait complete. 2022-12-01T09:47:44.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-12-01T09:47:35.244202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-12-01T09:47:42.293080+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-12-01T09:47:44.058 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-12-01T09:47:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.177+0000 7f6bbc760700 1 Processor -- start 2022-12-01T09:47:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.177+0000 7f6bbc760700 1 -- start start 2022-12-01T09:47:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.178+0000 7f6bbc760700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb40b5b60 0x7f6bb406d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.178+0000 7f6bbc760700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb40cc7a0 con 0x7f6bb4159710 2022-12-01T09:47:44.185 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.178+0000 7f6bbc760700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb40cc1e0 con 0x7f6bb406d540 2022-12-01T09:47:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.178+0000 7f6bbc760700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb406d9c0 con 0x7f6bb40b5b60 2022-12-01T09:47:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.178+0000 7f6bba4fc700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6bb4159710 0x7f6bb40b53f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52662/0 (socket says 172.21.15.84:52662) 2022-12-01T09:47:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.178+0000 7f6bba4fc700 1 -- 172.21.15.84:0/2097543562 learned_addr learned my addr 172.21.15.84:0/2097543562 (peer_addr_for_me v1:172.21.15.84:0/0) 2022-12-01T09:47:44.187 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.178+0000 7f6bb9cfb700 1 --2- 172.21.15.84:0/2097543562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb40b5b60 0x7f6bb406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.187 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb94fa700 1 -- 172.21.15.84:0/2097543562 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 225082712 0 0) 0x7f6bb40cc7a0 con 0x7f6bb4159710 2022-12-01T09:47:44.188 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb94fa700 1 -- 172.21.15.84:0/2097543562 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b9c0035b0 con 0x7f6bb4159710 2022-12-01T09:47:44.188 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb94fa700 1 -- 172.21.15.84:0/2097543562 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784694991 0 0) 0x7f6bb40cc1e0 con 0x7f6bb406d540 2022-12-01T09:47:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb94fa700 1 -- 172.21.15.84:0/2097543562 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb40cc7a0 con 0x7f6bb406d540 2022-12-01T09:47:44.189 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb9cfb700 1 -- 172.21.15.84:0/2097543562 >> v1:172.21.15.191:6789/0 conn(0x7f6bb406d540 legacy=0x7f6bb40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb9cfb700 1 -- 172.21.15.84:0/2097543562 >> v1:172.21.15.84:6789/0 conn(0x7f6bb4159710 legacy=0x7f6bb40b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb9cfb700 1 -- 172.21.15.84:0/2097543562 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb40cc4c0 con 0x7f6bb40b5b60 2022-12-01T09:47:44.190 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb9cfb700 1 --2- 172.21.15.84:0/2097543562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb40b5b60 0x7f6bb406d000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453a45e6d49e75bc server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.191 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb94fa700 1 -- 172.21.15.84:0/2097543562 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0011830 con 0x7f6bb40b5b60 2022-12-01T09:47:44.191 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb94fa700 1 -- 172.21.15.84:0/2097543562 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb0011990 con 0x7f6bb40b5b60 2022-12-01T09:47:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.179+0000 7f6bb94fa700 1 -- 172.21.15.84:0/2097543562 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0011d20 con 0x7f6bb40b5b60 2022-12-01T09:47:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.180+0000 7f6bbc760700 1 -- 172.21.15.84:0/2097543562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb40b5b60 msgr2=0x7f6bb406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.192 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.180+0000 7f6bbc760700 1 --2- 172.21.15.84:0/2097543562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb40b5b60 0x7f6bb406d000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009870 tx=0x7f6bb0009b80 comp rx=0 tx=0).stop 2022-12-01T09:47:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.180+0000 7f6bbc760700 1 -- 172.21.15.84:0/2097543562 shutdown_connections 2022-12-01T09:47:44.193 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.180+0000 7f6bbc760700 1 --2- 172.21.15.84:0/2097543562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb40b5b60 0x7f6bb406d000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.180+0000 7f6bbc760700 1 -- 172.21.15.84:0/2097543562 >> 172.21.15.84:0/2097543562 conn(0x7f6bb40cff30 msgr2=0x7f6bb40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.180+0000 7f6bbc760700 1 -- 172.21.15.84:0/2097543562 shutdown_connections 2022-12-01T09:47:44.194 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.180+0000 7f6bbc760700 1 -- 172.21.15.84:0/2097543562 wait complete. 2022-12-01T09:47:44.195 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.181+0000 7f6bbc760700 1 Processor -- start 2022-12-01T09:47:44.195 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.181+0000 7f6bbc760700 1 -- start start 2022-12-01T09:47:44.199 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbc760700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406d540 0x7f6bb40afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.199 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbc760700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40b5b60 0x7f6bb40aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbc760700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb4159710 0x7f6bb414b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbc760700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb414bda0 con 0x7f6bb406d540 2022-12-01T09:47:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbc760700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41e0600 con 0x7f6bb40b5b60 2022-12-01T09:47:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbc760700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41e08e0 con 0x7f6bb4159710 2022-12-01T09:47:44.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406d540 0x7f6bb40afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.201 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406d540 0x7f6bb40afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47818/0 (socket says 172.21.15.84:47818) 2022-12-01T09:47:44.201 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 -- 172.21.15.84:0/868986019 learned_addr learned my addr 172.21.15.84:0/868986019 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:44.201 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bb9cfb700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40b5b60 0x7f6bb40aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.201 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbacfd700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb4159710 0x7f6bb414b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 -- 172.21.15.84:0/868986019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb4159710 msgr2=0x7f6bb414b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb4159710 0x7f6bb414b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 -- 172.21.15.84:0/868986019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40b5b60 msgr2=0x7f6bb40aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40b5b60 0x7f6bb40aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 -- 172.21.15.84:0/868986019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00094a0 con 0x7f6bb406d540 2022-12-01T09:47:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bba4fc700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406d540 0x7f6bb40afc70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c700 tx=0x7f6ba400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6ba2ffd700 1 -- 172.21.15.84:0/868986019 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4005d70 con 0x7f6bb406d540 2022-12-01T09:47:44.203 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6ba2ffd700 1 -- 172.21.15.84:0/868986019 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba40070d0 con 0x7f6bb406d540 2022-12-01T09:47:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb4069430 con 0x7f6bb406d540 2022-12-01T09:47:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6ba2ffd700 1 -- 172.21.15.84:0/868986019 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4007440 con 0x7f6bb406d540 2022-12-01T09:47:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.182+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb40699f0 con 0x7f6bb406d540 2022-12-01T09:47:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.183+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb4067420 con 0x7f6bb406d540 2022-12-01T09:47:44.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.183+0000 7f6ba2ffd700 1 -- 172.21.15.84:0/868986019 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f6ba4019030 con 0x7f6bb406d540 2022-12-01T09:47:44.205 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.184+0000 7f6ba2ffd700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f6b94066590 0x7f6b94068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.205 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.184+0000 7f6ba2ffd700 1 -- 172.21.15.84:0/868986019 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6ba40825e0 con 0x7f6bb406d540 2022-12-01T09:47:44.205 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.184+0000 7f6bb9cfb700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f6b94066590 0x7f6b94068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.206 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.184+0000 7f6bb9cfb700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f6b94066590 0x7f6b94068a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00097c0 tx=0x7f6bb001c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.206 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.188+0000 7f6ba2ffd700 1 -- 172.21.15.84:0/868986019 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f6ba404abe0 con 0x7f6bb406d540 2022-12-01T09:47:44.370 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.365+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6bb406e330 con 0x7f6bb406d540 2022-12-01T09:47:44.370 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.366+0000 7f6ba2ffd700 1 -- 172.21.15.84:0/868986019 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ba4017020 con 0x7f6bb406d540 2022-12-01T09:47:44.371 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2022-12-01T09:47:44.372 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f6b94066590 msgr2=0x7f6b94068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.373 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f6b94066590 0x7f6b94068a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00097c0 tx=0x7f6bb001c040 comp rx=0 tx=0).stop 2022-12-01T09:47:44.373 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406d540 msgr2=0x7f6bb40afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.373 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406d540 0x7f6bb40afc70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c700 tx=0x7f6ba400ca10 comp rx=0 tx=0).stop 2022-12-01T09:47:44.373 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 shutdown_connections 2022-12-01T09:47:44.374 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6bb4159710 0x7f6bb414b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.374 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f6b94066590 0x7f6b94068a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.374 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6bb406d540 0x7f6bb40afc70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.374 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 --2- 172.21.15.84:0/868986019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bb40b5b60 0x7f6bb40aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.375 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 >> 172.21.15.84:0/868986019 conn(0x7f6bb40cff30 msgr2=0x7f6bb40b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:44.375 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 shutdown_connections 2022-12-01T09:47:44.375 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.368+0000 7f6bbc760700 1 -- 172.21.15.84:0/868986019 wait complete. 2022-12-01T09:47:44.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-12-01T09:47:44.506 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.501+0000 7f1e31330700 1 Processor -- start 2022-12-01T09:47:44.506 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.501+0000 7f1e31330700 1 -- start start 2022-12-01T09:47:44.510 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.502+0000 7f1e31330700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c159710 0x7f1e2c0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.510 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.502+0000 7f1e31330700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2c0cc7a0 con 0x7f1e2c0b5b60 2022-12-01T09:47:44.511 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.502+0000 7f1e31330700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2c0b5930 con 0x7f1e2c06d710 2022-12-01T09:47:44.511 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.502+0000 7f1e31330700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0cc1e0 con 0x7f1e2c159710 2022-12-01T09:47:44.511 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.502+0000 7f1e2a7fc700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1e2c0b5b60 0x7f1e2c06d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52676/0 (socket says 172.21.15.84:52676) 2022-12-01T09:47:44.511 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.502+0000 7f1e2a7fc700 1 -- 172.21.15.84:0/398690683 learned_addr learned my addr 172.21.15.84:0/398690683 (peer_addr_for_me v1:172.21.15.84:0/0) 2022-12-01T09:47:44.512 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.502+0000 7f1e2affd700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c159710 0x7f1e2c0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.512 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.502+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490641646 0 0) 0x7f1e2c0cc7a0 con 0x7f1e2c0b5b60 2022-12-01T09:47:44.512 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e180035b0 con 0x7f1e2c0b5b60 2022-12-01T09:47:44.512 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3600242694 0 0) 0x7f1e180035b0 con 0x7f1e2c0b5b60 2022-12-01T09:47:44.513 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c159710 msgr2=0x7f1e2c0b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:44.513 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c159710 0x7f1e2c0b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.513 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 >> v1:172.21.15.191:6789/0 conn(0x7f1e2c06d710 legacy=0x7f1e2c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.513 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0cc4c0 con 0x7f1e2c0b5b60 2022-12-01T09:47:44.513 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2125368202 0 0) 0x7f1e1c001670 con 0x7f1e2c0b5b60 2022-12-01T09:47:44.514 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 >> v1:172.21.15.84:6789/0 conn(0x7f1e2c0b5b60 legacy=0x7f1e2c06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.514 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e18003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.514 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e18004850 0x7f1e18004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.514 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c159710 0x7f1e18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.515 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0cc1e0 con 0x7f1e18003a30 2022-12-01T09:47:44.515 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1800bac0 con 0x7f1e18004850 2022-12-01T09:47:44.515 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1800bc50 con 0x7f1e2c159710 2022-12-01T09:47:44.515 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2affd700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e18003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.516 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2b7fe700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c159710 0x7f1e18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.516 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2a7fc700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e18004850 0x7f1e18004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.516 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2affd700 1 -- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c159710 msgr2=0x7f1e18005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:44.516 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2affd700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c159710 0x7f1e18005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.517 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2affd700 1 -- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e18004850 msgr2=0x7f1e18004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:44.517 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2affd700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e18004850 0x7f1e18004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.517 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2affd700 1 -- 172.21.15.84:0/398690683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e180035b0 con 0x7f1e18003a30 2022-12-01T09:47:44.517 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2a7fc700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e18004850 0x7f1e18004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:47:44.517 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e2affd700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e18003e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009b60 tx=0x7f1e18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dea655ad7a2b3f server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.518 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.503+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000e6c0 con 0x7f1e18003a30 2022-12-01T09:47:44.518 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2000e820 con 0x7f1e18003a30 2022-12-01T09:47:44.518 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e29ffb700 1 -- 172.21.15.84:0/398690683 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000eb90 con 0x7f1e18003a30 2022-12-01T09:47:44.518 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e31330700 1 -- 172.21.15.84:0/398690683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 msgr2=0x7f1e18003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.519 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e31330700 1 --2- 172.21.15.84:0/398690683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e18003e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009b60 tx=0x7f1e18005880 comp rx=0 tx=0).stop 2022-12-01T09:47:44.519 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e2b7fe700 1 -- 172.21.15.84:0/398690683 reap_dead start 2022-12-01T09:47:44.519 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e31330700 1 -- 172.21.15.84:0/398690683 shutdown_connections 2022-12-01T09:47:44.519 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e31330700 1 -- 172.21.15.84:0/398690683 >> 172.21.15.84:0/398690683 conn(0x7f1e2c0cff30 msgr2=0x7f1e2c0d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:44.520 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e31330700 1 -- 172.21.15.84:0/398690683 shutdown_connections 2022-12-01T09:47:44.520 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.504+0000 7f1e31330700 1 -- 172.21.15.84:0/398690683 wait complete. 2022-12-01T09:47:44.520 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.506+0000 7f1e31330700 1 Processor -- start 2022-12-01T09:47:44.520 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.506+0000 7f1e31330700 1 -- start start 2022-12-01T09:47:44.524 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e31330700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e2c14fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.524 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e31330700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c0c39b0 0x7f1e2c0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.524 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e31330700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e2c0c8c30 0x7f1e2c14f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.524 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e31330700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0c4c90 con 0x7f1e18003a30 2022-12-01T09:47:44.525 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e31330700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0c3f00 con 0x7f1e2c0c8c30 2022-12-01T09:47:44.525 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e31330700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c0c4360 con 0x7f1e2c0c39b0 2022-12-01T09:47:44.525 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e2c14fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.525 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e2c14fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47838/0 (socket says 172.21.15.84:47838) 2022-12-01T09:47:44.526 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 -- 172.21.15.84:0/2590608850 learned_addr learned my addr 172.21.15.84:0/2590608850 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:44.526 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2a7fc700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c0c39b0 0x7f1e2c0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.526 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2b7fe700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e2c0c8c30 0x7f1e2c14f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.526 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 -- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c0c39b0 msgr2=0x7f1e2c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.526 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c0c39b0 0x7f1e2c0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.527 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 -- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e2c0c8c30 msgr2=0x7f1e2c14f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.527 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e2c0c8c30 0x7f1e2c14f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.527 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 -- 172.21.15.84:0/2590608850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0c4640 con 0x7f1e18003a30 2022-12-01T09:47:44.527 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2a7fc700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c0c39b0 0x7f1e2c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:47:44.528 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2b7fe700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e2c0c8c30 0x7f1e2c14f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:47:44.528 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e2affd700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e2c14fe70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e180047a0 tx=0x7f1e200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.528 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e137fe700 1 -- 172.21.15.84:0/2590608850 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20021070 con 0x7f1e18003a30 2022-12-01T09:47:44.528 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e137fe700 1 -- 172.21.15.84:0/2590608850 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e200045d0 con 0x7f1e18003a30 2022-12-01T09:47:44.529 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e137fe700 1 -- 172.21.15.84:0/2590608850 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2000e6c0 con 0x7f1e18003a30 2022-12-01T09:47:44.529 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.507+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2c0c4920 con 0x7f1e18003a30 2022-12-01T09:47:44.529 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.508+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c14f790 con 0x7f1e18003a30 2022-12-01T09:47:44.529 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.508+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2c067420 con 0x7f1e18003a30 2022-12-01T09:47:44.529 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.509+0000 7f1e137fe700 1 -- 172.21.15.84:0/2590608850 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f1e2002e030 con 0x7f1e18003a30 2022-12-01T09:47:44.530 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.509+0000 7f1e137fe700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f1e0c066590 0x7f1e0c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.530 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.509+0000 7f1e137fe700 1 -- 172.21.15.84:0/2590608850 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1e200846d0 con 0x7f1e18003a30 2022-12-01T09:47:44.530 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.509+0000 7f1e2a7fc700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f1e0c066590 0x7f1e0c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.530 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.509+0000 7f1e2a7fc700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f1e0c066590 0x7f1e0c068a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c001eb0 tx=0x7f1e1c001dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.531 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.513+0000 7f1e137fe700 1 -- 172.21.15.84:0/2590608850 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f1e20052130 con 0x7f1e18003a30 2022-12-01T09:47:44.698 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.694+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e2c0c4c90 con 0x7f1e18003a30 2022-12-01T09:47:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.694+0000 7f1e137fe700 1 -- 172.21.15.84:0/2590608850 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e20055750 con 0x7f1e18003a30 2022-12-01T09:47:44.699 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2022-12-01T09:47:44.701 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f1e0c066590 msgr2=0x7f1e0c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.701 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f1e0c066590 0x7f1e0c068a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c001eb0 tx=0x7f1e1c001dc0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.701 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 msgr2=0x7f1e2c14fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.701 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e2c14fe70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e180047a0 tx=0x7f1e200047c0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 shutdown_connections 2022-12-01T09:47:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e2c0c8c30 0x7f1e2c14f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f1e0c066590 0x7f1e0c068a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e18003a30 0x7f1e2c14fe70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 --2- 172.21.15.84:0/2590608850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e2c0c39b0 0x7f1e2c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 >> 172.21.15.84:0/2590608850 conn(0x7f1e2c0cff30 msgr2=0x7f1e2c0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 shutdown_connections 2022-12-01T09:47:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.696+0000 7f1e31330700 1 -- 172.21.15.84:0/2590608850 wait complete. 2022-12-01T09:47:44.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-12-01T09:47:44.834 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.830+0000 7f06b54ea700 1 Processor -- start 2022-12-01T09:47:44.835 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.830+0000 7f06b54ea700 1 -- start start 2022-12-01T09:47:44.838 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06b54ea700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f06b015fdc0 0x7f06b00cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.838 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06b54ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b00d1540 0x7f06b00d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.838 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06b54ea700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b00cc950 con 0x7f06b00b3ae0 2022-12-01T09:47:44.839 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06b54ea700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b00cc6b0 con 0x7f06b015fdc0 2022-12-01T09:47:44.839 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06b54ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b00ccde0 con 0x7f06b00d1540 2022-12-01T09:47:44.839 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06af7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b00d1540 0x7f06b00d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.839 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06ae7fc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f06b015fdc0 0x7f06b00cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.840 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06af7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b00d1540 0x7f06b00d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.84:45616/0 (socket says 172.21.15.84:45616) 2022-12-01T09:47:44.840 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.831+0000 7f06af7fe700 1 -- 172.21.15.84:0/2058724075 learned_addr learned my addr 172.21.15.84:0/2058724075 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:44.840 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06af7fe700 1 -- 172.21.15.84:0/2058724075 >> v2:172.21.15.191:3300/0 conn(0x7f06b015fdc0 msgr2=0x7f06b00cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:44.841 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06af7fe700 1 --2- 172.21.15.84:0/2058724075 >> v2:172.21.15.191:3300/0 conn(0x7f06b015fdc0 0x7f06b00cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.841 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06af7fe700 1 -- 172.21.15.84:0/2058724075 >> v1:172.21.15.191:6789/0 conn(0x7f06b00b3ae0 legacy=0x7f06b00b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.841 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06af7fe700 1 -- 172.21.15.84:0/2058724075 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b01602c0 con 0x7f06b00d1540 2022-12-01T09:47:44.841 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06ae7fc700 1 --2- 172.21.15.84:0/2058724075 >> v2:172.21.15.191:3300/0 conn(0x7f06b015fdc0 0x7f06b00cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-12-01T09:47:44.842 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06af7fe700 1 --2- 172.21.15.84:0/2058724075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b00d1540 0x7f06b00d0170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f06a00098f0 tx=0x7f06a0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc75fffcdc6dcb server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.842 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06adffb700 1 -- 172.21.15.84:0/2058724075 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0011830 con 0x7f06b00d1540 2022-12-01T09:47:44.842 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06adffb700 1 -- 172.21.15.84:0/2058724075 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a0011990 con 0x7f06b00d1540 2022-12-01T09:47:44.842 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06adffb700 1 -- 172.21.15.84:0/2058724075 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0011d20 con 0x7f06b00d1540 2022-12-01T09:47:44.843 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06b54ea700 1 -- 172.21.15.84:0/2058724075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b00d1540 msgr2=0x7f06b00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.843 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06b54ea700 1 --2- 172.21.15.84:0/2058724075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b00d1540 0x7f06b00d0170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f06a00098f0 tx=0x7f06a0009c00 comp rx=0 tx=0).stop 2022-12-01T09:47:44.843 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06b54ea700 1 -- 172.21.15.84:0/2058724075 shutdown_connections 2022-12-01T09:47:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06b54ea700 1 --2- 172.21.15.84:0/2058724075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b00d1540 0x7f06b00d0170 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06b54ea700 1 --2- 172.21.15.84:0/2058724075 >> v2:172.21.15.191:3300/0 conn(0x7f06b015fdc0 0x7f06b00cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.832+0000 7f06b54ea700 1 -- 172.21.15.84:0/2058724075 >> 172.21.15.84:0/2058724075 conn(0x7f06b00ceae0 msgr2=0x7f06b00ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.833+0000 7f06b54ea700 1 -- 172.21.15.84:0/2058724075 shutdown_connections 2022-12-01T09:47:44.844 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.833+0000 7f06b54ea700 1 -- 172.21.15.84:0/2058724075 wait complete. 2022-12-01T09:47:44.845 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.834+0000 7f06b54ea700 1 Processor -- start 2022-12-01T09:47:44.845 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.834+0000 7f06b54ea700 1 -- start start 2022-12-01T09:47:44.848 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.834+0000 7f06b54ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06b00b3ae0 0x7f06b00c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.848 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.834+0000 7f06b54ea700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06b00d1540 0x7f06b0153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.849 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.834+0000 7f06b54ea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b015fdc0 0x7f06b00b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.849 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06b54ea700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b00c70f0 con 0x7f06b00b3ae0 2022-12-01T09:47:44.849 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06b54ea700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b00c5a60 con 0x7f06b00d1540 2022-12-01T09:47:44.849 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06b54ea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b00c5d40 con 0x7f06b015fdc0 2022-12-01T09:47:44.850 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06b00b3ae0 0x7f06b00c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.850 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06b00b3ae0 0x7f06b00c68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47854/0 (socket says 172.21.15.84:47854) 2022-12-01T09:47:44.850 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 -- 172.21.15.84:0/1576748615 learned_addr learned my addr 172.21.15.84:0/1576748615 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:44.850 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06af7fe700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b015fdc0 0x7f06b00b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.850 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06ae7fc700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06b00d1540 0x7f06b0153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.851 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 -- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b015fdc0 msgr2=0x7f06b00b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.851 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b015fdc0 0x7f06b00b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.851 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 -- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06b00d1540 msgr2=0x7f06b0153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:44.851 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06b00d1540 0x7f06b0153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:44.852 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 -- 172.21.15.84:0/1576748615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980094a0 con 0x7f06b00b3ae0 2022-12-01T09:47:44.852 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06af7fe700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b015fdc0 0x7f06b00b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:47:44.852 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06ae7fc700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06b00d1540 0x7f06b0153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:47:44.852 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06aeffd700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06b00b3ae0 0x7f06b00c68a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06a800cbd0 tx=0x7f06a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.852 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06a77fe700 1 -- 172.21.15.84:0/1576748615 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8003c00 con 0x7f06b00b3ae0 2022-12-01T09:47:44.853 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06a77fe700 1 -- 172.21.15.84:0/1576748615 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8003d60 con 0x7f06b00b3ae0 2022-12-01T09:47:44.853 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a0009550 con 0x7f06b00b3ae0 2022-12-01T09:47:44.853 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06a77fe700 1 -- 172.21.15.84:0/1576748615 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800abf0 con 0x7f06b00b3ae0 2022-12-01T09:47:44.853 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.835+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b00c73d0 con 0x7f06b00b3ae0 2022-12-01T09:47:44.854 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.836+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b00c6260 con 0x7f06b00b3ae0 2022-12-01T09:47:44.854 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.837+0000 7f06a77fe700 1 -- 172.21.15.84:0/1576748615 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f06a8010050 con 0x7f06b00b3ae0 2022-12-01T09:47:44.854 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.837+0000 7f06a77fe700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0694066590 0x7f0694068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:44.854 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.837+0000 7f06ae7fc700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0694066590 0x7f0694068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:44.855 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.837+0000 7f06a77fe700 1 -- 172.21.15.84:0/1576748615 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f06a8083410 con 0x7f06b00b3ae0 2022-12-01T09:47:44.855 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.837+0000 7f06ae7fc700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0694066590 0x7f0694068a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06b01601f0 tx=0x7f0698008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:44.855 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:44.841+0000 7f06a77fe700 1 -- 172.21.15.84:0/1576748615 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f06b00c6260 con 0x7f06b00b3ae0 2022-12-01T09:47:45.068 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.064+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06b00c6260 con 0x7f06b00b3ae0 2022-12-01T09:47:45.068 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.064+0000 7f06a77fe700 1 -- 172.21.15.84:0/1576748615 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f06a8050e70 con 0x7f06b00b3ae0 2022-12-01T09:47:45.069 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:47:45.069 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/3515473688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3515473688},{"type":"v1","addr":"172.21.15.84:6836","nonce":3515473688}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/812369925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":812369925},{"type":"v1","addr":"172.21.15.84:6837","nonce":812369925}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/745792619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":745792619},{"type":"v1","addr":"172.21.15.191:6835","nonce":745792619}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2033088117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2033088117},{"type":"v1","addr":"172.21.15.191:6834","nonce":2033088117}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-12-01T09:47:45.072 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0694066590 msgr2=0x7f0694068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.072 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0694066590 0x7f0694068a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06b01601f0 tx=0x7f0698008040 comp rx=0 tx=0).stop 2022-12-01T09:47:45.073 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06b00b3ae0 msgr2=0x7f06b00c68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.073 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06b00b3ae0 0x7f06b00c68a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06a800cbd0 tx=0x7f06a800cf90 comp rx=0 tx=0).stop 2022-12-01T09:47:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 shutdown_connections 2022-12-01T09:47:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06b015fdc0 0x7f06b00b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.074 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0694066590 0x7f0694068a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.075 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06b00b3ae0 0x7f06b00c68a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.075 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 --2- 172.21.15.84:0/1576748615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06b00d1540 0x7f06b0153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 >> 172.21.15.84:0/1576748615 conn(0x7f06b00ceae0 msgr2=0x7f06b015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 shutdown_connections 2022-12-01T09:47:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.066+0000 7f06b54ea700 1 -- 172.21.15.84:0/1576748615 wait complete. 2022-12-01T09:47:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 3 2022-12-01T09:47:45.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-12-01T09:47:45.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.197+0000 7f4e7f756700 1 Processor -- start 2022-12-01T09:47:45.203 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.198+0000 7f4e7f756700 1 -- start start 2022-12-01T09:47:45.214 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.198+0000 7f4e7f756700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4e7815b9d0 0x7f4e7815fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.215 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.198+0000 7f4e7f756700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780cc940 0x7f4e78071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.215 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.198+0000 7f4e7f756700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4e780ae3f0 0x7f4e780724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.216 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.198+0000 7f4e7f756700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7815bda0 con 0x7f4e780ae3f0 2022-12-01T09:47:45.216 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.198+0000 7f4e7f756700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e7815f7f0 con 0x7f4e7815b9d0 2022-12-01T09:47:45.217 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.198+0000 7f4e7f756700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780cceb0 con 0x7f4e780cc940 2022-12-01T09:47:45.217 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4e780ae3f0 0x7f4e780724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.218 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4e780ae3f0 0x7f4e780724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47866/0 (socket says 172.21.15.84:47866) 2022-12-01T09:47:45.218 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 -- 172.21.15.84:0/655047460 learned_addr learned my addr 172.21.15.84:0/655047460 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:45.219 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7ccf1700 1 --2- 172.21.15.84:0/655047460 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780cc940 0x7f4e78071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.219 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7d4f2700 1 --2- 172.21.15.84:0/655047460 >> v2:172.21.15.191:3300/0 conn(0x7f4e7815b9d0 0x7f4e7815fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.220 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 -- 172.21.15.84:0/655047460 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780cc940 msgr2=0x7f4e78071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:45.220 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 --2- 172.21.15.84:0/655047460 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780cc940 0x7f4e78071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.221 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 -- 172.21.15.84:0/655047460 >> v2:172.21.15.191:3300/0 conn(0x7f4e7815b9d0 msgr2=0x7f4e7815fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:45.221 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 --2- 172.21.15.84:0/655047460 >> v2:172.21.15.191:3300/0 conn(0x7f4e7815b9d0 0x7f4e7815fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.222 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 -- 172.21.15.84:0/655047460 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e780ce2f0 con 0x7f4e780ae3f0 2022-12-01T09:47:45.222 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e7dcf3700 1 --2- 172.21.15.84:0/655047460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780ae3f0 0x7f4e780724a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e780ceb20 tx=0x7f4e7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a5833fa8042802 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.223 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e67fff700 1 -- 172.21.15.84:0/655047460 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7000e070 con 0x7f4e780ae3f0 2022-12-01T09:47:45.223 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e67fff700 1 -- 172.21.15.84:0/655047460 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7000b910 con 0x7f4e780ae3f0 2022-12-01T09:47:45.224 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.199+0000 7f4e67fff700 1 -- 172.21.15.84:0/655047460 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7000bcc0 con 0x7f4e780ae3f0 2022-12-01T09:47:45.224 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 -- 172.21.15.84:0/655047460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780ae3f0 msgr2=0x7f4e780724a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.225 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 --2- 172.21.15.84:0/655047460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780ae3f0 0x7f4e780724a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e780ceb20 tx=0x7f4e7000b330 comp rx=0 tx=0).stop 2022-12-01T09:47:45.225 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 -- 172.21.15.84:0/655047460 shutdown_connections 2022-12-01T09:47:45.225 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 --2- 172.21.15.84:0/655047460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e780ae3f0 0x7f4e780724a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.226 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 --2- 172.21.15.84:0/655047460 >> v2:172.21.15.191:3300/0 conn(0x7f4e7815b9d0 0x7f4e7815fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.226 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 --2- 172.21.15.84:0/655047460 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780cc940 0x7f4e78071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.227 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 -- 172.21.15.84:0/655047460 >> 172.21.15.84:0/655047460 conn(0x7f4e780cb7f0 msgr2=0x7f4e780cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:45.227 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 -- 172.21.15.84:0/655047460 shutdown_connections 2022-12-01T09:47:45.228 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.200+0000 7f4e7f756700 1 -- 172.21.15.84:0/655047460 wait complete. 2022-12-01T09:47:45.228 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.201+0000 7f4e7f756700 1 Processor -- start 2022-12-01T09:47:45.228 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.201+0000 7f4e7f756700 1 -- start start 2022-12-01T09:47:45.229 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7f756700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780ae3f0 0x7f4e780b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.229 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7f756700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e780cc940 0x7f4e780b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.230 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7f756700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e7815b9d0 0x7f4e78070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.230 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7f756700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780b27e0 con 0x7f4e7815b9d0 2022-12-01T09:47:45.230 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7f756700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7806c500 con 0x7f4e780cc940 2022-12-01T09:47:45.231 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7f756700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780b42d0 con 0x7f4e780ae3f0 2022-12-01T09:47:45.231 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e7815b9d0 0x7f4e78070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.232 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e7815b9d0 0x7f4e78070620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47870/0 (socket says 172.21.15.84:47870) 2022-12-01T09:47:45.232 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 -- 172.21.15.84:0/2670329245 learned_addr learned my addr 172.21.15.84:0/2670329245 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:45.232 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7d4f2700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780ae3f0 0x7f4e780b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.233 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7ccf1700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e780cc940 0x7f4e780b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.233 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 -- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780ae3f0 msgr2=0x7f4e780b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.233 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780ae3f0 0x7f4e780b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.233 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 -- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e780cc940 msgr2=0x7f4e780b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.234 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e780cc940 0x7f4e780b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.234 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 -- 172.21.15.84:0/2670329245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70009b30 con 0x7f4e7815b9d0 2022-12-01T09:47:45.234 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7ccf1700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e780cc940 0x7f4e780b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-12-01T09:47:45.234 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7dcf3700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e7815b9d0 0x7f4e78070620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e78071890 tx=0x7f4e70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.234 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e65ffb700 1 -- 172.21.15.84:0/2670329245 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e700099b0 con 0x7f4e7815b9d0 2022-12-01T09:47:45.235 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e65ffb700 1 -- 172.21.15.84:0/2670329245 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e700092e0 con 0x7f4e7815b9d0 2022-12-01T09:47:45.235 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.202+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e780b45b0 con 0x7f4e7815b9d0 2022-12-01T09:47:45.235 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.203+0000 7f4e65ffb700 1 -- 172.21.15.84:0/2670329245 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004470 con 0x7f4e7815b9d0 2022-12-01T09:47:45.235 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.203+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e780b4b70 con 0x7f4e7815b9d0 2022-12-01T09:47:45.236 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.204+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7806c670 con 0x7f4e7815b9d0 2022-12-01T09:47:45.236 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.204+0000 7f4e65ffb700 1 -- 172.21.15.84:0/2670329245 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f4e7002c070 con 0x7f4e7815b9d0 2022-12-01T09:47:45.236 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.204+0000 7f4e65ffb700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f4e58066590 0x7f4e58068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.236 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.204+0000 7f4e65ffb700 1 -- 172.21.15.84:0/2670329245 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4e700848a0 con 0x7f4e7815b9d0 2022-12-01T09:47:45.236 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.204+0000 7f4e7d4f2700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f4e58066590 0x7f4e58068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.237 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.205+0000 7f4e7d4f2700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f4e58066590 0x7f4e58068a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006b20 tx=0x7f4e68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.237 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.208+0000 7f4e65ffb700 1 -- 172.21.15.84:0/2670329245 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f4e70052190 con 0x7f4e7815b9d0 2022-12-01T09:47:45.394 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.389+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e78067420 con 0x7f4e7815b9d0 2022-12-01T09:47:45.394 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.390+0000 7f4e65ffb700 1 -- 172.21.15.84:0/2670329245 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13741 (secure 0 0 0) 0x7f4e70020090 con 0x7f4e7815b9d0 2022-12-01T09:47:45.395 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:47:45.395 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":19,"fsid":"8329cc4d-9cf5-499b-b4c9-a9793681da24","created":"2022-12-01T09:47:23.909978+0000","modified":"2022-12-01T09:47:44.685340+0000","last_up_change":"2022-12-01T09:47:35.194323+0000","last_in_change":"2022-12-01T09:47:26.403178+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-12-01T09:47:35.244202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-12-01T09:47:42.293080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"e9c852ba-df6e-4c09-98f8-692b089e63c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6801","nonce":101363},{"type":"v1","addr":"172.21.15.84:6803","nonce":101363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6805","nonce":101363},{"type":"v1","addr":"172.21.15.84:6807","nonce":101363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6813","nonce":101363},{"type":"v1","addr":"172.21.15.84:6815","nonce":101363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6809","nonce":101363},{"type":"v1","addr":"172.21.15.84:6811","nonce":101363}]},"public_addr":"172.21.15.84:6803/101363","cluster_addr":"172.21.15.84:6807/101363","heartbeat_back_addr":"172.21.15.84:6815/101363","heartbeat_front_addr":"172.21.15.84:6811/101363","state":["exists","up"]},{"osd":1,"uuid":"2b47da52-0666-49d8-b263-725119e42db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":101364},{"type":"v1","addr":"172.21.15.84:6825","nonce":101364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":101364},{"type":"v1","addr":"172.21.15.84:6827","nonce":101364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":101364},{"type":"v1","addr":"172.21.15.84:6831","nonce":101364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":101364},{"type":"v1","addr":"172.21.15.84:6829","nonce":101364}]},"public_addr":"172.21.15.84:6825/101364","cluster_addr":"172.21.15.84:6827/101364","heartbeat_back_addr":"172.21.15.84:6831/101364","heartbeat_front_addr":"172.21.15.84:6829/101364","state":["exists","up"]},{"osd":2,"uuid":"7b668a53-55d9-481c-83a5-df590cd91fad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":101362},{"type":"v1","addr":"172.21.15.84:6802","nonce":101362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":101362},{"type":"v1","addr":"172.21.15.84:6806","nonce":101362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":101362},{"type":"v1","addr":"172.21.15.84:6814","nonce":101362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":101362},{"type":"v1","addr":"172.21.15.84:6810","nonce":101362}]},"public_addr":"172.21.15.84:6802/101362","cluster_addr":"172.21.15.84:6806/101362","heartbeat_back_addr":"172.21.15.84:6814/101362","heartbeat_front_addr":"172.21.15.84:6810/101362","state":["exists","up"]},{"osd":3,"uuid":"1f2c87e3-7398-4f50-a56c-5f798d2fb279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":101365},{"type":"v1","addr":"172.21.15.84:6817","nonce":101365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":101365},{"type":"v1","addr":"172.21.15.84:6819","nonce":101365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":101365},{"type":"v1","addr":"172.21.15.84:6823","nonce":101365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":101365},{"type":"v1","addr":"172.21.15.84:6821","nonce":101365}]},"public_addr":"172.21.15.84:6817/101365","cluster_addr":"172.21.15.84:6819/101365","heartbeat_back_addr":"172.21.15.84:6823/101365","heartbeat_front_addr":"172.21.15.84:6821/101365","state":["exists","up"]},{"osd":4,"uuid":"8a4b43a4-6c63-48ab-826a-cb8919c44b8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":101621},{"type":"v1","addr":"172.21.15.191:6801","nonce":101621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":101621},{"type":"v1","addr":"172.21.15.191:6803","nonce":101621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":101621},{"type":"v1","addr":"172.21.15.191:6807","nonce":101621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":101621},{"type":"v1","addr":"172.21.15.191:6805","nonce":101621}]},"public_addr":"172.21.15.191:6801/101621","cluster_addr":"172.21.15.191:6803/101621","heartbeat_back_addr":"172.21.15.191:6807/101621","heartbeat_front_addr":"172.21.15.191:6805/101621","state":["exists","up"]},{"osd":5,"uuid":"18a3160f-c761-49e7-8d7d-56fdf4c8a7f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":101635},{"type":"v1","addr":"172.21.15.191:6817","nonce":101635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":101635},{"type":"v1","addr":"172.21.15.191:6819","nonce":101635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":101635},{"type":"v1","addr":"172.21.15.191:6823","nonce":101635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":101635},{"type":"v1","addr":"172.21.15.191:6821","nonce":101635}]},"public_addr":"172.21.15.191:6817/101635","cluster_addr":"172.21.15.191:6819/101635","heartbeat_back_addr":"172.21.15.191:6823/101635","heartbeat_front_addr":"172.21.15.191:6821/101635","state":["exists","up"]},{"osd":6,"uuid":"e70b8522-3456-479c-81c9-10e06ad3172f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":101622},{"type":"v1","addr":"172.21.15.191:6809","nonce":101622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":101622},{"type":"v1","addr":"172.21.15.191:6811","nonce":101622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":101622},{"type":"v1","addr":"172.21.15.191:6815","nonce":101622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":101622},{"type":"v1","addr":"172.21.15.191:6813","nonce":101622}]},"public_addr":"172.21.15.191:6809/101622","cluster_addr":"172.21.15.191:6811/101622","heartbeat_back_addr":"172.21.15.191:6815/101622","heartbeat_front_addr":"172.21.15.191:6813/101622","state":["exists","up"]},{"osd":7,"uuid":"d2c33094-6303-4ed3-963c-e010df6aa820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":101646},{"type":"v1","addr":"172.21.15.191:6825","nonce":101646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":101646},{"type":"v1","addr":"172.21.15.191:6827","nonce":101646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":101646},{"type":"v1","addr":"172.21.15.191:6831","nonce":101646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":101646},{"type":"v1","addr":"172.21.15.191:6829","nonce":101646}]},"public_addr":"172.21.15.191:6825/101646","cluster_addr":"172.21.15.191:6827/101646","heartbeat_back_addr":"172.21.15.191:6831/101646","heartbeat_front_addr":"172.21.15.191:6829/101646","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.585643+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.563418+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.596853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.531044+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.550829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.604675+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.514679+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-12-01T09:47:31.557085+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-12-01T09:47:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f4e58066590 msgr2=0x7f4e58068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f4e58066590 0x7f4e58068a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e68006b20 tx=0x7f4e68008040 comp rx=0 tx=0).stop 2022-12-01T09:47:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e7815b9d0 msgr2=0x7f4e78070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.397 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e7815b9d0 0x7f4e78070620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e78071890 tx=0x7f4e70004ad0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 shutdown_connections 2022-12-01T09:47:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f4e58066590 0x7f4e58068a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e7815b9d0 0x7f4e78070620 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.398 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e780ae3f0 0x7f4e780b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 --2- 172.21.15.84:0/2670329245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e780cc940 0x7f4e780b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 >> 172.21.15.84:0/2670329245 conn(0x7f4e780cb7f0 msgr2=0x7f4e781537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 shutdown_connections 2022-12-01T09:47:45.399 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.392+0000 7f4e7f756700 1 -- 172.21.15.84:0/2670329245 wait complete. 2022-12-01T09:47:45.409 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-12-01T09:47:35.244202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-12-01T09:47:42.293080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-12-01T09:47:45.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-12-01T09:47:45.533 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.528+0000 7fd97ebf7700 1 Processor -- start 2022-12-01T09:47:45.533 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.528+0000 7fd97ebf7700 1 -- start start 2022-12-01T09:47:45.536 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd97ebf7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9780ad540 0x7fd9780ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.536 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd97ebf7700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd9780c9220 0x7fd9780abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.536 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd97ebf7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd978158c30 con 0x7fd978158860 2022-12-01T09:47:45.537 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd97ebf7700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97815d080 con 0x7fd9780c9220 2022-12-01T09:47:45.537 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd97ebf7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9780ab090 con 0x7fd9780ad540 2022-12-01T09:47:45.537 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd97d194700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd9780c9220 0x7fd9780abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.537 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd97d194700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd9780c9220 0x7fd9780abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47880/0 (socket says 172.21.15.84:47880) 2022-12-01T09:47:45.538 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd97d194700 1 -- 172.21.15.84:0/3470199028 learned_addr learned my addr 172.21.15.84:0/3470199028 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:45.538 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.529+0000 7fd977fff700 1 --2- 172.21.15.84:0/3470199028 >> v2:172.21.15.191:3300/0 conn(0x7fd9780ad540 0x7fd9780ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.538 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97d194700 1 -- 172.21.15.84:0/3470199028 >> v2:172.21.15.191:3300/0 conn(0x7fd9780ad540 msgr2=0x7fd9780ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:45.538 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97d194700 1 --2- 172.21.15.84:0/3470199028 >> v2:172.21.15.191:3300/0 conn(0x7fd9780ad540 0x7fd9780ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.539 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97d194700 1 -- 172.21.15.84:0/3470199028 >> v1:172.21.15.191:6789/0 conn(0x7fd978158860 legacy=0x7fd9780c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:45.539 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97d194700 1 -- 172.21.15.84:0/3470199028 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780ab370 con 0x7fd9780c9220 2022-12-01T09:47:45.539 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97d194700 1 --2- 172.21.15.84:0/3470199028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9780c9220 0x7fd9780abe50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd97806f050 tx=0x7fd97000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8826a8172632bc server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.539 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd9777fe700 1 -- 172.21.15.84:0/3470199028 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97000e070 con 0x7fd9780c9220 2022-12-01T09:47:45.540 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd9777fe700 1 -- 172.21.15.84:0/3470199028 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd970003a20 con 0x7fd9780c9220 2022-12-01T09:47:45.540 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd9777fe700 1 -- 172.21.15.84:0/3470199028 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970003dd0 con 0x7fd9780c9220 2022-12-01T09:47:45.540 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97ebf7700 1 -- 172.21.15.84:0/3470199028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9780c9220 msgr2=0x7fd9780abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.541 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/3470199028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9780c9220 0x7fd9780abe50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd97806f050 tx=0x7fd97000b330 comp rx=0 tx=0).stop 2022-12-01T09:47:45.541 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97ebf7700 1 -- 172.21.15.84:0/3470199028 shutdown_connections 2022-12-01T09:47:45.541 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/3470199028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9780c9220 0x7fd9780abe50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.542 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/3470199028 >> v2:172.21.15.191:3300/0 conn(0x7fd9780ad540 0x7fd9780ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.542 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97ebf7700 1 -- 172.21.15.84:0/3470199028 >> 172.21.15.84:0/3470199028 conn(0x7fd978070a30 msgr2=0x7fd978070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:45.542 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97ebf7700 1 -- 172.21.15.84:0/3470199028 shutdown_connections 2022-12-01T09:47:45.542 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.530+0000 7fd97ebf7700 1 -- 172.21.15.84:0/3470199028 wait complete. 2022-12-01T09:47:45.543 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.532+0000 7fd97ebf7700 1 Processor -- start 2022-12-01T09:47:45.543 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.532+0000 7fd97ebf7700 1 -- start start 2022-12-01T09:47:45.547 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.532+0000 7fd97ebf7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9780ad540 0x7fd9780b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.547 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.532+0000 7fd97ebf7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9780c9220 0x7fd9780b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.547 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.532+0000 7fd97ebf7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd978158860 0x7fd9780c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.548 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.532+0000 7fd97ebf7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780ae2b0 con 0x7fd978158860 2022-12-01T09:47:45.548 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.532+0000 7fd97ebf7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780c3730 con 0x7fd9780c9220 2022-12-01T09:47:45.548 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.532+0000 7fd97ebf7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9780c3a10 con 0x7fd9780ad540 2022-12-01T09:47:45.549 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd978158860 0x7fd9780c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.549 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd978158860 0x7fd9780c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47886/0 (socket says 172.21.15.84:47886) 2022-12-01T09:47:45.549 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 -- 172.21.15.84:0/689712885 learned_addr learned my addr 172.21.15.84:0/689712885 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:45.549 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97c993700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9780ad540 0x7fd9780b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.550 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd977fff700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9780c9220 0x7fd9780b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.550 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 -- 172.21.15.84:0/689712885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9780ad540 msgr2=0x7fd9780b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.550 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9780ad540 0x7fd9780b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.551 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 -- 172.21.15.84:0/689712885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9780c9220 msgr2=0x7fd9780b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.551 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9780c9220 0x7fd9780b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.551 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 -- 172.21.15.84:0/689712885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd970009b30 con 0x7fd978158860 2022-12-01T09:47:45.551 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97d194700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd978158860 0x7fd9780c2a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b370 tx=0x7fd970004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.552 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd9757fa700 1 -- 172.21.15.84:0/689712885 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970004440 con 0x7fd978158860 2022-12-01T09:47:45.552 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd9757fa700 1 -- 172.21.15.84:0/689712885 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9700045c0 con 0x7fd978158860 2022-12-01T09:47:45.552 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9780c3cf0 con 0x7fd978158860 2022-12-01T09:47:45.552 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd9757fa700 1 -- 172.21.15.84:0/689712885 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970004970 con 0x7fd978158860 2022-12-01T09:47:45.553 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.533+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9780b0770 con 0x7fd978158860 2022-12-01T09:47:45.553 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.534+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9780ae4b0 con 0x7fd978158860 2022-12-01T09:47:45.553 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.535+0000 7fd9757fa700 1 -- 172.21.15.84:0/689712885 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fd970022030 con 0x7fd978158860 2022-12-01T09:47:45.553 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.535+0000 7fd9757fa700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fd960066590 0x7fd960068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.554 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.535+0000 7fd9757fa700 1 -- 172.21.15.84:0/689712885 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd970084510 con 0x7fd978158860 2022-12-01T09:47:45.554 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.535+0000 7fd97c993700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fd960066590 0x7fd960068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.555 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.535+0000 7fd97c993700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fd960066590 0x7fd960068a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006fd0 tx=0x7fd96c0090e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.555 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.539+0000 7fd9757fa700 1 -- 172.21.15.84:0/689712885 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7fd9780ae4b0 con 0x7fd978158860 2022-12-01T09:47:45.723 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.718+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd978158c30 con 0x7fd978158860 2022-12-01T09:47:45.724 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.719+0000 7fd9757fa700 1 -- 172.21.15.84:0/689712885 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd978158c30 con 0x7fd978158860 2022-12-01T09:47:45.724 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2022-12-01T09:47:45.726 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fd960066590 msgr2=0x7fd960068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.726 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fd960066590 0x7fd960068a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006fd0 tx=0x7fd96c0090e0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.726 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd978158860 msgr2=0x7fd9780c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd978158860 0x7fd9780c2a00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd97000b370 tx=0x7fd970004ad0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 shutdown_connections 2022-12-01T09:47:45.727 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7fd960066590 0x7fd960068a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.728 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd978158860 0x7fd9780c2a00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.728 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9780ad540 0x7fd9780b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.728 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 --2- 172.21.15.84:0/689712885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9780c9220 0x7fd9780b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.729 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 >> 172.21.15.84:0/689712885 conn(0x7fd978070a30 msgr2=0x7fd9780ac230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:45.729 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 shutdown_connections 2022-12-01T09:47:45.729 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.721+0000 7fd97ebf7700 1 -- 172.21.15.84:0/689712885 wait complete. 2022-12-01T09:47:45.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-12-01T09:47:45.861 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.856+0000 7f99b6c0a700 1 Processor -- start 2022-12-01T09:47:45.861 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b6c0a700 1 -- start start 2022-12-01T09:47:45.866 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b6c0a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99b00cd350 0x7f99b00cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b6c0a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00c44f0 0x7f99b00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b6c0a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b00cdc60 con 0x7f99b00cb3e0 2022-12-01T09:47:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b6c0a700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b00c9e00 con 0x7f99b00cd350 2022-12-01T09:47:45.867 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b6c0a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00ca0e0 con 0x7f99b00c44f0 2022-12-01T09:47:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b49a6700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99b00cd350 0x7f99b00cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b49a6700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99b00cd350 0x7f99b00cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47890/0 (socket says 172.21.15.84:47890) 2022-12-01T09:47:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.857+0000 7f99b49a6700 1 -- 172.21.15.84:0/382965682 learned_addr learned my addr 172.21.15.84:0/382965682 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:45.869 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99affff700 1 --2- 172.21.15.84:0/382965682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00c44f0 0x7f99b00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.869 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99b49a6700 1 -- 172.21.15.84:0/382965682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00c44f0 msgr2=0x7f99b00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:45.869 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99b49a6700 1 --2- 172.21.15.84:0/382965682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00c44f0 0x7f99b00caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.869 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99b49a6700 1 -- 172.21.15.84:0/382965682 >> v1:172.21.15.191:6789/0 conn(0x7f99b00cb3e0 legacy=0x7f99b0069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99b49a6700 1 -- 172.21.15.84:0/382965682 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b00ca3c0 con 0x7f99b00cd350 2022-12-01T09:47:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99b49a6700 1 --2- 172.21.15.84:0/382965682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00cd350 0x7f99b00cd720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99a400bb90 tx=0x7f99a400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d837fea92783c7c6 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99af7fe700 1 -- 172.21.15.84:0/382965682 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a400dd10 con 0x7f99b00cd350 2022-12-01T09:47:45.870 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99af7fe700 1 -- 172.21.15.84:0/382965682 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a400de70 con 0x7f99b00cd350 2022-12-01T09:47:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99af7fe700 1 -- 172.21.15.84:0/382965682 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4004230 con 0x7f99b00cd350 2022-12-01T09:47:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99b6c0a700 1 -- 172.21.15.84:0/382965682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00cd350 msgr2=0x7f99b00cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.858+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/382965682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00cd350 0x7f99b00cd720 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99a400bb90 tx=0x7f99a400bea0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.871 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.859+0000 7f99b6c0a700 1 -- 172.21.15.84:0/382965682 shutdown_connections 2022-12-01T09:47:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.859+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/382965682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00cd350 0x7f99b00cd720 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.859+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/382965682 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00c44f0 0x7f99b00caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.859+0000 7f99b6c0a700 1 -- 172.21.15.84:0/382965682 >> 172.21.15.84:0/382965682 conn(0x7f99b00cb7f0 msgr2=0x7f99b00cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:45.872 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.859+0000 7f99b6c0a700 1 -- 172.21.15.84:0/382965682 shutdown_connections 2022-12-01T09:47:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.859+0000 7f99b6c0a700 1 -- 172.21.15.84:0/382965682 wait complete. 2022-12-01T09:47:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.860+0000 7f99b6c0a700 1 Processor -- start 2022-12-01T09:47:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.860+0000 7f99b6c0a700 1 -- start start 2022-12-01T09:47:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b6c0a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00c44f0 0x7f99b00c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.873 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b6c0a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00cb3e0 0x7f99b014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.874 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b6c0a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00cd350 0x7f99b00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.874 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b6c0a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00b3270 con 0x7f99b00c44f0 2022-12-01T09:47:45.874 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b6c0a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b014ed40 con 0x7f99b00cb3e0 2022-12-01T09:47:45.874 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b6c0a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00c6d60 con 0x7f99b00cd350 2022-12-01T09:47:45.875 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00c44f0 0x7f99b00c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.875 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00c44f0 0x7f99b00c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47896/0 (socket says 172.21.15.84:47896) 2022-12-01T09:47:45.875 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 -- 172.21.15.84:0/436305989 learned_addr learned my addr 172.21.15.84:0/436305989 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:45.876 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b51a7700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00cd350 0x7f99b00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.876 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99affff700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00cb3e0 0x7f99b014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.876 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 -- 172.21.15.84:0/436305989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00cd350 msgr2=0x7f99b00c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.876 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00cd350 0x7f99b00c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 -- 172.21.15.84:0/436305989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00cb3e0 msgr2=0x7f99b014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00cb3e0 0x7f99b014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 -- 172.21.15.84:0/436305989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a400b7f0 con 0x7f99b00c44f0 2022-12-01T09:47:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b49a6700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00c44f0 0x7f99b00c7fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a400b340 tx=0x7f99a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.878 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99ad7fa700 1 -- 172.21.15.84:0/436305989 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4003ea0 con 0x7f99b00c44f0 2022-12-01T09:47:45.878 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99ad7fa700 1 -- 172.21.15.84:0/436305989 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a400dcc0 con 0x7f99b00c44f0 2022-12-01T09:47:45.878 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b00c7030 con 0x7f99b00c44f0 2022-12-01T09:47:45.878 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99ad7fa700 1 -- 172.21.15.84:0/436305989 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40106b0 con 0x7f99b00c44f0 2022-12-01T09:47:45.879 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.861+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b00c75f0 con 0x7f99b00c44f0 2022-12-01T09:47:45.879 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.862+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b0067420 con 0x7f99b00c44f0 2022-12-01T09:47:45.880 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.863+0000 7f99ad7fa700 1 -- 172.21.15.84:0/436305989 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f99a402c460 con 0x7f99b00c44f0 2022-12-01T09:47:45.881 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.864+0000 7f99ad7fa700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9998066590 0x7f9998068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:45.881 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.864+0000 7f99ad7fa700 1 -- 172.21.15.84:0/436305989 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f99a40854f0 con 0x7f99b00c44f0 2022-12-01T09:47:45.881 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.864+0000 7f99affff700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9998066590 0x7f9998068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:45.881 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.864+0000 7f99affff700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9998066590 0x7f9998068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99b006df10 tx=0x7f999c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:45.882 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:45.867+0000 7f99ad7fa700 1 -- 172.21.15.84:0/436305989 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f99a4052f90 con 0x7f99b00c44f0 2022-12-01T09:47:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.045+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f99b00ca6a0 con 0x7f99b00c44f0 2022-12-01T09:47:46.050 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.045+0000 7f99ad7fa700 1 -- 172.21.15.84:0/436305989 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f99a40565b0 con 0x7f99b00c44f0 2022-12-01T09:47:46.050 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2022-12-01T09:47:46.052 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9998066590 msgr2=0x7f9998068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:46.052 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9998066590 0x7f9998068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99b006df10 tx=0x7f999c008040 comp rx=0 tx=0).stop 2022-12-01T09:47:46.052 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00c44f0 msgr2=0x7f99b00c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:46.053 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00c44f0 0x7f99b00c7fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99a400b340 tx=0x7f99a4004750 comp rx=0 tx=0).stop 2022-12-01T09:47:46.053 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 shutdown_connections 2022-12-01T09:47:46.053 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99b00cd350 0x7f99b00c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.053 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f9998066590 0x7f9998068a40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.054 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99b00c44f0 0x7f99b00c7fb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.054 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 --2- 172.21.15.84:0/436305989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99b00cb3e0 0x7f99b014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.054 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 >> 172.21.15.84:0/436305989 conn(0x7f99b00cb7f0 msgr2=0x7f99b006a510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:46.054 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 shutdown_connections 2022-12-01T09:47:46.055 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.047+0000 7f99b6c0a700 1 -- 172.21.15.84:0/436305989 wait complete. 2022-12-01T09:47:46.063 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-12-01T09:47:46.183 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.179+0000 7f0bca9e3700 1 Processor -- start 2022-12-01T09:47:46.184 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.179+0000 7f0bca9e3700 1 -- start start 2022-12-01T09:47:46.187 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.179+0000 7f0bca9e3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc415b9d0 0x7f0bc415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:46.187 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.179+0000 7f0bca9e3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0bc40cc940 0x7f0bc4071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:46.188 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bca9e3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc415f7f0 con 0x7f0bc40ae3f0 2022-12-01T09:47:46.188 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bca9e3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc415bda0 con 0x7f0bc40cc940 2022-12-01T09:47:46.188 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bca9e3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40cceb0 con 0x7f0bc415b9d0 2022-12-01T09:47:46.188 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc415b9d0 0x7f0bc415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:46.188 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0bc40cc940 0x7f0bc4071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:46.189 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc415b9d0 0x7f0bc415fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.84:45668/0 (socket says 172.21.15.84:45668) 2022-12-01T09:47:46.189 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc3fff700 1 -- 172.21.15.84:0/996854349 learned_addr learned my addr 172.21.15.84:0/996854349 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:46.189 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc3fff700 1 -- 172.21.15.84:0/996854349 >> v2:172.21.15.191:3300/0 conn(0x7f0bc40cc940 msgr2=0x7f0bc4071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-12-01T09:47:46.190 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc3fff700 1 --2- 172.21.15.84:0/996854349 >> v2:172.21.15.191:3300/0 conn(0x7f0bc40cc940 0x7f0bc4071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.190 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc3fff700 1 -- 172.21.15.84:0/996854349 >> v1:172.21.15.191:6789/0 conn(0x7f0bc40ae3f0 legacy=0x7f0bc4069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:46.190 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc3fff700 1 -- 172.21.15.84:0/996854349 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40ce2f0 con 0x7f0bc415b9d0 2022-12-01T09:47:46.190 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.180+0000 7f0bc3fff700 1 --2- 172.21.15.84:0/996854349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc415b9d0 0x7f0bc415fc80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098f0 tx=0x7f0bb8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c14746075775370 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bc2ffd700 1 -- 172.21.15.84:0/996854349 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8011830 con 0x7f0bc415b9d0 2022-12-01T09:47:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bc2ffd700 1 -- 172.21.15.84:0/996854349 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb8011990 con 0x7f0bc415b9d0 2022-12-01T09:47:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bc2ffd700 1 -- 172.21.15.84:0/996854349 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8011d20 con 0x7f0bc415b9d0 2022-12-01T09:47:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bca9e3700 1 -- 172.21.15.84:0/996854349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc415b9d0 msgr2=0x7f0bc415fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/996854349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc415b9d0 0x7f0bc415fc80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80098f0 tx=0x7f0bb8009c00 comp rx=0 tx=0).stop 2022-12-01T09:47:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bca9e3700 1 -- 172.21.15.84:0/996854349 shutdown_connections 2022-12-01T09:47:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/996854349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc415b9d0 0x7f0bc415fc80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/996854349 >> v2:172.21.15.191:3300/0 conn(0x7f0bc40cc940 0x7f0bc4071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.192 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bca9e3700 1 -- 172.21.15.84:0/996854349 >> 172.21.15.84:0/996854349 conn(0x7f0bc40cb7f0 msgr2=0x7f0bc40cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-12-01T09:47:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bca9e3700 1 -- 172.21.15.84:0/996854349 shutdown_connections 2022-12-01T09:47:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.181+0000 7f0bca9e3700 1 -- 172.21.15.84:0/996854349 wait complete. 2022-12-01T09:47:46.193 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.182+0000 7f0bca9e3700 1 Processor -- start 2022-12-01T09:47:46.194 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.183+0000 7f0bca9e3700 1 -- start start 2022-12-01T09:47:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.183+0000 7f0bca9e3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc40ae3f0 0x7f0bc4159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.183+0000 7f0bca9e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc40cc940 0x7f0bc40b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:46.197 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.183+0000 7f0bca9e3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc415b9d0 0x7f0bc41f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.183+0000 7f0bca9e3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41f5a90 con 0x7f0bc40cc940 2022-12-01T09:47:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.183+0000 7f0bca9e3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc40b2030 con 0x7f0bc415b9d0 2022-12-01T09:47:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.183+0000 7f0bca9e3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc41f4e80 con 0x7f0bc40ae3f0 2022-12-01T09:47:46.198 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc40cc940 0x7f0bc40b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:46.199 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc40cc940 0x7f0bc40b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47912/0 (socket says 172.21.15.84:47912) 2022-12-01T09:47:46.199 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 -- 172.21.15.84:0/2247243838 learned_addr learned my addr 172.21.15.84:0/2247243838 (peer_addr_for_me v2:172.21.15.84:0/0) 2022-12-01T09:47:46.199 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc3fff700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc40ae3f0 0x7f0bc4159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:46.199 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc8f80700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc415b9d0 0x7f0bc41f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:46.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 -- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc40ae3f0 msgr2=0x7f0bc4159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:46.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc40ae3f0 0x7f0bc4159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 -- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc415b9d0 msgr2=0x7f0bc41f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:46.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc415b9d0 0x7f0bc41f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.200 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 -- 172.21.15.84:0/2247243838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8009550 con 0x7f0bc40cc940 2022-12-01T09:47:46.201 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc8f80700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc415b9d0 0x7f0bc41f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:47:46.201 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc3fff700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc40ae3f0 0x7f0bc4159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-12-01T09:47:46.201 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc37fe700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc40cc940 0x7f0bc40b4720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9b0 tx=0x7f0bb400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:46.201 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc0ff9700 1 -- 172.21.15.84:0/2247243838 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4009710 con 0x7f0bc40cc940 2022-12-01T09:47:46.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc0ff9700 1 -- 172.21.15.84:0/2247243838 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb400c850 con 0x7f0bc40cc940 2022-12-01T09:47:46.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bca9e3700 1 -- 172.21.15.84:0/2247243838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc41f5160 con 0x7f0bc40cc940 2022-12-01T09:47:46.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bc0ff9700 1 -- 172.21.15.84:0/2247243838 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400cbc0 con 0x7f0bc40cc940 2022-12-01T09:47:46.202 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.184+0000 7f0bca9e3700 1 -- 172.21.15.84:0/2247243838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc41f5720 con 0x7f0bc40cc940 2022-12-01T09:47:46.203 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.185+0000 7f0bca9e3700 1 -- 172.21.15.84:0/2247243838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc4073070 con 0x7f0bc40cc940 2022-12-01T09:47:46.203 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.185+0000 7f0bc0ff9700 1 -- 172.21.15.84:0/2247243838 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f0bb4019030 con 0x7f0bc40cc940 2022-12-01T09:47:46.203 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.186+0000 7f0bc0ff9700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0ba4066590 0x7f0ba4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-12-01T09:47:46.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.186+0000 7f0bc0ff9700 1 -- 172.21.15.84:0/2247243838 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0bb4083430 con 0x7f0bc40cc940 2022-12-01T09:47:46.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.186+0000 7f0bc3fff700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0ba4066590 0x7f0ba4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-12-01T09:47:46.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.186+0000 7f0bc3fff700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0ba4066590 0x7f0ba4068a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8014040 tx=0x7f0bb8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-12-01T09:47:46.204 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.190+0000 7f0bc0ff9700 1 -- 172.21.15.84:0/2247243838 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170950 (secure 0 0 0) 0x7f0bb4050e90 con 0x7f0bc40cc940 2022-12-01T09:47:46.377 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.372+0000 7f0bca9e3700 1 -- 172.21.15.84:0/2247243838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0bc4067420 con 0x7f0bc40cc940 2022-12-01T09:47:46.377 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.373+0000 7f0bc0ff9700 1 -- 172.21.15.84:0/2247243838 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f0bb40544b0 con 0x7f0bc40cc940 2022-12-01T09:47:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: 2022-12-01T09:47:46.378 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2022-12-01T09:47:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: { 2022-12-01T09:47:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2022-12-01T09:47:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2022-12-01T09:47:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2022-12-01T09:47:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: { 2022-12-01T09:47:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 2, 2022-12-01T09:47:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "rbd" 2022-12-01T09:47:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: } 2022-12-01T09:47:46.381 INFO:teuthology.orchestra.run.smithi084.stdout:] 2022-12-01T09:47:46.383 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 -- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0ba4066590 msgr2=0x7f0ba4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:46.383 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0ba4066590 0x7f0ba4068a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8014040 tx=0x7f0bb8008030 comp rx=0 tx=0).stop 2022-12-01T09:47:46.383 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 -- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc40cc940 msgr2=0x7f0bc40b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-12-01T09:47:46.384 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc40cc940 0x7f0bc40b4720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9b0 tx=0x7f0bb400ed70 comp rx=0 tx=0).stop 2022-12-01T09:47:46.384 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 -- 172.21.15.84:0/2247243838 shutdown_connections 2022-12-01T09:47:46.384 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0bc415b9d0 0x7f0bc41f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.384 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0bc40ae3f0 0x7f0bc4159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.385 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:6832/101090,v1:172.21.15.84:6833/101090] conn(0x7f0ba4066590 0x7f0ba4068a40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.385 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 --2- 172.21.15.84:0/2247243838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bc40cc940 0x7f0bc40b4720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-12-01T09:47:46.385 INFO:teuthology.orchestra.run.smithi084.stderr:2022-12-01T09:47:46.375+0000 7f0bca9e3700 1 -- 172.21.15.84:0/2247243838 >> 172.21.15.84:0/2247243838 conn(0x7f0bc40cb7f0 msgr2=0x7f0bc41