2022-11-27T14:23:34.419 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-27T14:23:34.421 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088565 --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} --owner scheduled_yuriw@teuthology --verbose --name yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi -- /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088565/orig.config.yaml 2022-11-27T14:23:34.469 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-27T14:23:34.546 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088565 branch: main description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} email: ceph-qa@ceph.io first_in_suite: false job_id: '7088565' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 sleep_before_teardown: 0 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAp1TozT0uutM8HOCPzxEUSXZmhnkJgIflw9VEshFT4NgBcl8qb5zeruryssm+11PL0ZzY0QFAJKRCeRNotEZM= smithi137.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0i848JIwZQdAX9/JFm15VYV34au3I21waYzUFCIZXYbgTm1YraDScyXY6UBdStOjwsIGb0D5/dgJK1MOwpAdcyXzZC2VzgxxhCTVxjrttdvpTM0LXq5ZEUxcdPgInjAH93llCZdUvVRr6a7KA2KUDHEN9CjQ5TgrgIoV+GsJN5w7JM8eJfMUBEaabd2Plp36YccCUbtxf8BgRYkcZCU+D8zODIEonvdQoIelNsTOwOP8qZq3xLLHCbniHc3ulWZGq6RqfyMGxkoJeqHI+e4X9YMe0qHz2Vl/Lubu4xwlUyFzoge19FREG2Uas5syv+0AmQz1c0OK44W57Iy28IARekPOdPm0TaaaA6cJEcCh1+D7wTqZG1kIEZBdqfoxoMXkDUSAqsvY/meDVnTgAQvE9yiadF1QIjz/1k9RhiYhZA3CeRfOQsYtYTBpz+DB0uz9ZvjOkegY2bSZbB39LagOk+73kB4KSijezWCjOn3S8I/QBt9lnJ75PqZIsknJWulc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_multifs_auth teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-23_17:24:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.11998 2022-11-27T14:23:34.547 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa; will attempt to use it 2022-11-27T14:23:34.548 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa/tasks 2022-11-27T14:23:34.548 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-27T14:23:34.551 INFO:teuthology.task.internal:Checking packages... 2022-11-27T14:23:34.581 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '697b0d99d9a52df92b20c8e053dcd46c75f364a9' 2022-11-27T14:23:34.582 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T14:23:34.583 INFO:teuthology.packaging:ref: None 2022-11-27T14:23:34.583 INFO:teuthology.packaging:tag: None 2022-11-27T14:23:34.583 INFO:teuthology.packaging:branch: main 2022-11-27T14:23:34.584 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:23:34.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2022-11-27T14:23:34.822 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-982.g2bb1aa26 2022-11-27T14:23:34.822 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-27T14:23:34.830 INFO:teuthology.task.internal:no buildpackages task found 2022-11-27T14:23:34.830 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-27T14:23:34.847 INFO:teuthology.task.internal:Saving configuration 2022-11-27T14:23:34.867 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-27T14:23:34.873 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-27T14:23:34.898 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088565', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 14:18:28.004075', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAp1TozT0uutM8HOCPzxEUSXZmhnkJgIflw9VEshFT4NgBcl8qb5zeruryssm+11PL0ZzY0QFAJKRCeRNotEZM='} 2022-11-27T14:23:34.923 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088565', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 14:18:28.002313', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMeqDJM18yPeV+aIr84d+Gk/foijtB3dDpni2GIZVGqigU1vjPtVfUM/SSHFKKPjtiQNpto3yMIRZKrwyusf+js='} 2022-11-27T14:23:34.923 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-27T14:23:34.930 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2022-11-27T14:23:34.930 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2022-11-27T14:23:34.930 INFO:teuthology.run_tasks:Running task console_log... 2022-11-27T14:23:35.017 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f695aae6158>, signals=[15]) 2022-11-27T14:23:35.018 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-27T14:23:35.024 INFO:teuthology.task.internal:Opening connections... 2022-11-27T14:23:35.024 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2022-11-27T14:23:35.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:23:35.146 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2022-11-27T14:23:35.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:23:35.234 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-27T14:23:35.243 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2022-11-27T14:23:35.265 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2022-11-27T14:23:35.265 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2022-11-27T14:23:35.325 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T14:23:35.326 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T14:23:35.326 INFO:teuthology.orchestra.run.smithi099.stdout:ID="rhel" 2022-11-27T14:23:35.326 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="fedora" 2022-11-27T14:23:35.326 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8.6" 2022-11-27T14:23:35.327 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2022-11-27T14:23:35.327 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T14:23:35.327 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2022-11-27T14:23:35.327 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T14:23:35.327 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T14:23:35.328 INFO:teuthology.orchestra.run.smithi099.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T14:23:35.328 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T14:23:35.328 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:23:35.328 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T14:23:35.329 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T14:23:35.329 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T14:23:35.329 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T14:23:35.330 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2022-11-27T14:23:35.355 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2022-11-27T14:23:35.375 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2022-11-27T14:23:35.376 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2022-11-27T14:23:35.434 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T14:23:35.435 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T14:23:35.435 INFO:teuthology.orchestra.run.smithi137.stdout:ID="rhel" 2022-11-27T14:23:35.435 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="fedora" 2022-11-27T14:23:35.436 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8.6" 2022-11-27T14:23:35.436 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8" 2022-11-27T14:23:35.436 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T14:23:35.436 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2022-11-27T14:23:35.437 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T14:23:35.437 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T14:23:35.437 INFO:teuthology.orchestra.run.smithi137.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T14:23:35.437 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T14:23:35.438 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:23:35.438 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T14:23:35.438 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T14:23:35.438 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T14:23:35.438 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T14:23:35.439 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2022-11-27T14:23:35.459 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-27T14:23:35.471 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-27T14:23:35.480 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-27T14:23:35.481 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2022-11-27T14:23:35.484 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2022-11-27T14:23:35.504 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-27T14:23:35.511 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-27T14:23:35.512 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2022-11-27T14:23:35.544 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2022-11-27T14:23:35.573 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-27T14:23:35.631 INFO:teuthology.run_tasks:Running task kernel... 2022-11-27T14:23:35.645 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2022-11-27T14:23:35.646 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2022-11-27T14:23:35.647 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2022-11-27T14:23:35.647 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T14:23:35.648 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T14:23:35.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:23:35.668 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2022-11-27T14:23:35.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:23:35.669 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2022-11-27T14:23:35.727 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T14:23:35.727 INFO:teuthology.task.kernel:Running kernel on smithi099: 4.18.0-372.9.1.el8.x86_64 2022-11-27T14:23:35.728 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum install -y kernel 2022-11-27T14:23:35.729 INFO:teuthology.orchestra.run.smithi137.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T14:23:35.729 INFO:teuthology.task.kernel:Running kernel on smithi137: 4.18.0-372.9.1.el8.x86_64 2022-11-27T14:23:35.730 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2022-11-27T14:23:36.760 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2022-11-27T14:23:36.760 INFO:teuthology.orchestra.run.smithi137.stdout:Unable to read consumer identity 2022-11-27T14:23:36.764 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2022-11-27T14:23:36.764 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2022-11-27T14:23:36.832 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:23:36.833 INFO:teuthology.orchestra.run.smithi137.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T14:23:36.833 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:23:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:23:36.860 INFO:teuthology.orchestra.run.smithi099.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T14:23:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:23:41.056 INFO:teuthology.orchestra.run.smithi137.stdout:Copr repo for python3-asyncssh owned by ceph 1.1 kB/s | 3.5 kB 00:03 2022-11-27T14:23:42.025 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2022-11-27T14:23:42.094 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 863 B/s | 3.5 kB 00:04 2022-11-27T14:23:43.122 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:01 2022-11-27T14:23:44.980 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 436 kB/s | 24 kB 00:00 2022-11-27T14:23:45.043 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:01 ago on Sun 27 Nov 2022 02:23:44 PM UTC. 2022-11-27T14:23:45.666 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T14:23:45.713 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2022-11-27T14:23:45.714 INFO:teuthology.orchestra.run.smithi137.stdout:Nothing to do. 2022-11-27T14:23:45.714 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2022-11-27T14:23:45.870 DEBUG:teuthology.orchestra.run.smithi137:> echo no | sudo yum reinstall kernel || true 2022-11-27T14:23:46.093 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 441 kB/s | 24 kB 00:00 2022-11-27T14:23:46.206 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2022-11-27T14:23:46.207 INFO:teuthology.orchestra.run.smithi137.stdout:Unable to read consumer identity 2022-11-27T14:23:46.264 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:23:46.264 INFO:teuthology.orchestra.run.smithi137.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T14:23:46.264 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:23:46.511 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 02:23:44 PM UTC. 2022-11-27T14:23:46.552 INFO:teuthology.orchestra.run.smithi137.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T14:23:46.604 INFO:teuthology.orchestra.run.smithi137.stderr:Error: No packages marked for reinstall. 2022-11-27T14:23:46.650 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum reinstall -y kernel || true 2022-11-27T14:23:46.762 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T14:23:46.808 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2022-11-27T14:23:46.809 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2022-11-27T14:23:46.809 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2022-11-27T14:23:46.959 DEBUG:teuthology.orchestra.run.smithi099:> echo no | sudo yum reinstall kernel || true 2022-11-27T14:23:46.983 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2022-11-27T14:23:46.984 INFO:teuthology.orchestra.run.smithi137.stdout:Unable to read consumer identity 2022-11-27T14:23:47.023 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:23:47.024 INFO:teuthology.orchestra.run.smithi137.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T14:23:47.024 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:23:47.257 INFO:teuthology.orchestra.run.smithi137.stdout:Last metadata expiration check: 0:00:03 ago on Sun 27 Nov 2022 02:23:44 PM UTC. 2022-11-27T14:23:47.295 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2022-11-27T14:23:47.296 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2022-11-27T14:23:47.297 INFO:teuthology.orchestra.run.smithi137.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T14:23:47.339 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:23:47.340 INFO:teuthology.orchestra.run.smithi099.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T14:23:47.340 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:23:47.348 INFO:teuthology.orchestra.run.smithi137.stderr:Error: No packages marked for reinstall. 2022-11-27T14:23:47.395 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel --last | head -n 1 2022-11-27T14:23:47.474 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T14:23:47.475 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T14:23:47.476 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T14:23:47.476 DEBUG:teuthology.parallel:result is None 2022-11-27T14:23:47.578 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:01 ago on Sun 27 Nov 2022 02:23:46 PM UTC. 2022-11-27T14:23:47.618 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T14:23:47.664 INFO:teuthology.orchestra.run.smithi099.stderr:Error: No packages marked for reinstall. 2022-11-27T14:23:47.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum reinstall -y kernel || true 2022-11-27T14:23:48.041 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2022-11-27T14:23:48.042 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2022-11-27T14:23:48.091 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:23:48.091 INFO:teuthology.orchestra.run.smithi099.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T14:23:48.091 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:23:48.331 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 02:23:46 PM UTC. 2022-11-27T14:23:48.371 INFO:teuthology.orchestra.run.smithi099.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T14:23:48.400 INFO:teuthology.orchestra.run.smithi099.stderr:Error: No packages marked for reinstall. 2022-11-27T14:23:48.450 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel --last | head -n 1 2022-11-27T14:23:48.530 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T14:23:48.532 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T14:23:48.532 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T14:23:48.533 DEBUG:teuthology.parallel:result is None 2022-11-27T14:23:48.533 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-27T14:23:48.543 INFO:teuthology.task.internal:Creating test directory... 2022-11-27T14:23:48.544 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T14:23:48.547 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T14:23:48.568 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-27T14:23:48.576 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-27T14:23:48.586 INFO:teuthology.task.internal:Creating archive directory... 2022-11-27T14:23:48.587 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T14:23:48.604 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T14:23:48.645 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-27T14:23:48.653 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-27T14:23:48.653 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T14:23:48.683 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T14:23:48.717 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:23:48.717 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:23:48.732 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:23:48.735 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T14:23:48.746 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-27T14:23:48.757 INFO:teuthology.task.internal:Configuring sudo... 2022-11-27T14:23:48.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T14:23:48.776 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T14:23:48.815 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-27T14:23:48.824 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-27T14:23:48.825 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T14:23:48.844 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T14:23:48.874 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:23:48.943 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:23:48.998 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:23:49.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:23:49.123 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:23:49.124 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T14:23:49.190 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:23:49.237 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T14:23:49.332 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:23:49.377 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T14:23:49.423 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:23:49.423 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T14:23:49.492 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2022-11-27T14:23:49.494 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2022-11-27T14:23:49.550 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T14:23:49.590 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T14:23:50.023 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-27T14:23:50.033 INFO:teuthology.task.internal:Starting timer... 2022-11-27T14:23:50.033 INFO:teuthology.run_tasks:Running task pcp... 2022-11-27T14:23:50.060 INFO:teuthology.run_tasks:Running task selinux... 2022-11-27T14:23:50.070 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2022-11-27T14:23:50.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2022-11-27T14:23:50.132 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2022-11-27T14:23:50.133 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2022-11-27T14:23:50.192 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2022-11-27T14:23:50.194 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-27T14:23:50.195 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2022-11-27T14:23:50.225 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2022-11-27T14:23:50.225 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2022-11-27T14:23:50.247 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2022-11-27T14:23:50.247 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive'} 2022-11-27T14:23:50.248 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T14:23:50.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:23:50.292 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T14:23:50.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T14:23:50.321 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-27T14:23:50.321 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2022-11-27T14:23:50.360 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2022-11-27T14:23:50.389 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-27T14:23:50.399 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2022-11-27T14:23:50.400 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-27T14:23:50.427 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-27T14:23:50.429 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-27T14:31:47.901 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com')] 2022-11-27T14:31:47.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2022-11-27T14:31:47.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:31:47.995 DEBUG:teuthology.orchestra.run.smithi099:> true 2022-11-27T14:31:48.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2022-11-27T14:31:48.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T14:31:48.139 DEBUG:teuthology.orchestra.run.smithi137:> true 2022-11-27T14:31:48.214 INFO:teuthology.run_tasks:Running task clock... 2022-11-27T14:31:48.231 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-27T14:31:48.232 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T14:31:48.234 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T14:31:48.237 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T14:31:48.238 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T14:31:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T14:31:48.281 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T14:31:48.287 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T14:31:48.302 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T14:31:48.346 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2022-11-27T14:31:48.357 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2022-11-27T14:31:48.372 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T14:31:48.385 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2022-11-27T14:31:48.386 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T14:31:48.397 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2022-11-27T14:31:48.412 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T14:31:48.425 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T14:31:48.465 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2022-11-27T14:31:48.468 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T14:31:48.468 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2022-11-27T14:31:48.468 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:31:48.469 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:31:48.469 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:31:48.469 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:31:48.499 INFO:teuthology.orchestra.run.smithi137.stderr:bash: ntpq: command not found 2022-11-27T14:31:48.502 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T14:31:48.503 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2022-11-27T14:31:48.503 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:31:48.503 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:31:48.503 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:31:48.504 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T14:31:48.505 INFO:teuthology.run_tasks:Running task install... 2022-11-27T14:31:48.514 DEBUG:teuthology.task.install:project ceph 2022-11-27T14:31:48.514 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}} 2022-11-27T14:31:48.515 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'} 2022-11-27T14:31:48.515 INFO:teuthology.task.install:Using flavor: default 2022-11-27T14:31:48.522 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-27T14:31:48.523 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-11-27T14:31:48.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo lsb_release -is 2022-11-27T14:31:48.564 INFO:teuthology.orchestra.run.smithi099.stdout:RedHatEnterprise 2022-11-27T14:31:48.566 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T14:31:48.566 DEBUG:teuthology.orchestra.run.smithi137:> sudo lsb_release -is 2022-11-27T14:31:48.567 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T14:31:48.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo lsb_release -is 2022-11-27T14:31:48.610 INFO:teuthology.orchestra.run.smithi137.stdout:RedHatEnterprise 2022-11-27T14:31:48.611 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T14:31:48.611 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T14:31:48.612 DEBUG:teuthology.orchestra.run.smithi137:> sudo lsb_release -is 2022-11-27T14:31:48.646 INFO:teuthology.orchestra.run.smithi099.stdout:RedHatEnterprise 2022-11-27T14:31:48.647 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T14:31:48.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:31:48.692 INFO:teuthology.orchestra.run.smithi137.stdout:RedHatEnterprise 2022-11-27T14:31:48.694 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T14:31:48.694 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:31:48.808 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T14:31:48.809 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T14:31:48.840 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T14:31:48.841 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T14:31:48.998 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T14:31:48.998 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:31:48.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T14:31:49.007 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T14:31:49.007 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:31:49.007 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T14:31:49.026 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T14:31:49.027 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T14:31:49.034 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T14:31:49.035 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T14:31:49.093 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T14:31:49.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T14:31:49.171 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T14:31:49.178 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T14:31:49.236 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2022-11-27T14:31:49.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2022-11-27T14:31:49.245 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2022-11-27T14:31:49.246 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2022-11-27T14:31:49.609 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2022-11-27T14:31:49.616 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2022-11-27T14:31:49.994 INFO:teuthology.orchestra.run.smithi137.stdout:54 files removed 2022-11-27T14:31:50.042 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T14:31:50.372 INFO:teuthology.orchestra.run.smithi137.stdout:Updating Subscription Management repositories. 2022-11-27T14:31:50.561 INFO:teuthology.orchestra.run.smithi099.stdout:54 files removed 2022-11-27T14:31:50.614 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T14:31:50.950 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2022-11-27T14:31:51.025 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 269 kB/s | 81 kB 00:00 2022-11-27T14:31:51.343 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2022-11-27T14:31:51.588 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 262 kB/s | 81 kB 00:00 2022-11-27T14:31:51.644 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 5.4 kB/s | 1.5 kB 00:00 2022-11-27T14:31:51.880 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2022-11-27T14:31:52.181 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 5.4 kB/s | 1.5 kB 00:00 2022-11-27T14:31:54.748 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T14:31:55.052 INFO:teuthology.orchestra.run.smithi137.stdout:Copr repo for python3-asyncssh owned by ceph 1.0 kB/s | 3.5 kB 00:03 2022-11-27T14:31:55.757 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2022-11-27T14:31:56.052 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:00 2022-11-27T14:31:58.707 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 419 kB/s | 24 kB 00:00 2022-11-27T14:31:58.970 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 350 kB/s | 24 kB 00:00 2022-11-27T14:31:59.551 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 48 MB 00:00 2022-11-27T14:31:59.772 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 65 MB/s | 48 MB 00:00 2022-11-27T14:32:07.568 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 70 MB/s | 53 MB 00:00 2022-11-27T14:32:07.764 INFO:teuthology.orchestra.run.smithi137.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 70 MB/s | 53 MB 00:00 2022-11-27T14:32:18.879 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T14:32:18.880 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T14:32:18.894 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T14:32:18.895 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T14:32:18.896 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T14:32:18.898 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T14:32:18.900 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T14:32:18.903 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T14:32:18.904 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T14:32:18.941 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T14:32:18.942 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T14:32:18.957 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T14:32:18.958 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T14:32:18.960 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T14:32:18.961 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T14:32:18.963 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T14:32:18.967 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T14:32:18.968 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T14:32:19.035 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2022-11-27T14:32:19.053 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================================================== 2022-11-27T14:32:19.053 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2022-11-27T14:32:19.054 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================================================== 2022-11-27T14:32:19.054 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2022-11-27T14:32:19.054 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T14:32:19.054 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T14:32:19.055 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T14:32:19.055 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T14:32:19.055 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T14:32:19.055 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T14:32:19.056 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T14:32:19.056 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T14:32:19.056 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T14:32:19.056 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T14:32:19.057 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T14:32:19.057 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T14:32:19.057 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T14:32:19.057 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T14:32:19.057 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T14:32:19.058 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T14:32:19.058 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T14:32:19.058 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T14:32:19.058 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T14:32:19.059 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T14:32:19.059 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T14:32:19.059 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T14:32:19.059 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T14:32:19.059 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T14:32:19.060 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T14:32:19.060 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T14:32:19.060 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T14:32:19.060 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T14:32:19.061 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T14:32:19.061 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T14:32:19.061 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T14:32:19.061 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T14:32:19.061 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T14:32:19.062 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T14:32:19.062 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T14:32:19.062 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T14:32:19.063 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T14:32:19.063 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2022-11-27T14:32:19.063 INFO:teuthology.orchestra.run.smithi137.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T14:32:19.064 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T14:32:19.064 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T14:32:19.065 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T14:32:19.065 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T14:32:19.065 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2022-11-27T14:32:19.066 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T14:32:19.067 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T14:32:19.067 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T14:32:19.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T14:32:19.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T14:32:19.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T14:32:19.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T14:32:19.068 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T14:32:19.069 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T14:32:19.069 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T14:32:19.069 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T14:32:19.070 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T14:32:19.070 INFO:teuthology.orchestra.run.smithi137.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T14:32:19.070 INFO:teuthology.orchestra.run.smithi137.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T14:32:19.071 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T14:32:19.071 INFO:teuthology.orchestra.run.smithi137.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T14:32:19.071 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T14:32:19.071 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T14:32:19.071 INFO:teuthology.orchestra.run.smithi137.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T14:32:19.072 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T14:32:19.072 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T14:32:19.072 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T14:32:19.073 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T14:32:19.073 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T14:32:19.073 INFO:teuthology.orchestra.run.smithi137.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T14:32:19.073 INFO:teuthology.orchestra.run.smithi137.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T14:32:19.074 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T14:32:19.074 INFO:teuthology.orchestra.run.smithi137.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T14:32:19.074 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T14:32:19.074 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T14:32:19.074 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T14:32:19.075 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T14:32:19.075 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T14:32:19.075 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T14:32:19.076 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T14:32:19.076 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T14:32:19.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T14:32:19.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T14:32:19.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T14:32:19.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T14:32:19.077 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T14:32:19.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T14:32:19.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T14:32:19.078 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T14:32:19.079 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T14:32:19.079 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T14:32:19.079 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T14:32:19.079 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T14:32:19.080 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T14:32:19.080 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T14:32:19.080 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T14:32:19.080 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T14:32:19.080 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T14:32:19.081 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T14:32:19.081 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T14:32:19.081 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T14:32:19.081 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T14:32:19.082 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T14:32:19.082 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T14:32:19.082 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T14:32:19.082 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T14:32:19.082 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T14:32:19.083 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T14:32:19.083 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T14:32:19.083 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T14:32:19.083 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T14:32:19.084 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T14:32:19.084 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T14:32:19.084 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T14:32:19.084 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T14:32:19.084 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T14:32:19.085 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T14:32:19.085 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T14:32:19.085 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T14:32:19.085 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T14:32:19.085 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T14:32:19.086 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T14:32:19.086 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T14:32:19.086 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T14:32:19.086 INFO:teuthology.orchestra.run.smithi137.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T14:32:19.087 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T14:32:19.087 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T14:32:19.087 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T14:32:19.087 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T14:32:19.088 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T14:32:19.088 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T14:32:19.088 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T14:32:19.089 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T14:32:19.089 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T14:32:19.089 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T14:32:19.090 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T14:32:19.090 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling module streams: 2022-11-27T14:32:19.090 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql 10 2022-11-27T14:32:19.090 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:32:19.090 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2022-11-27T14:32:19.091 INFO:teuthology.orchestra.run.smithi137.stdout:=========================================================================================================================== 2022-11-27T14:32:19.091 INFO:teuthology.orchestra.run.smithi137.stdout:Install 128 Packages 2022-11-27T14:32:19.091 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 5 Packages 2022-11-27T14:32:19.091 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:32:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 219 M 2022-11-27T14:32:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2022-11-27T14:32:19.101 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2022-11-27T14:32:19.113 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2022-11-27T14:32:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2022-11-27T14:32:19.114 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2022-11-27T14:32:19.114 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2022-11-27T14:32:19.114 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T14:32:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T14:32:19.115 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T14:32:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T14:32:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T14:32:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T14:32:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T14:32:19.116 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T14:32:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T14:32:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T14:32:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T14:32:19.117 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T14:32:19.118 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T14:32:19.118 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T14:32:19.118 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T14:32:19.118 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T14:32:19.118 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T14:32:19.119 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T14:32:19.119 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T14:32:19.119 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T14:32:19.119 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T14:32:19.120 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T14:32:19.120 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T14:32:19.120 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T14:32:19.121 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T14:32:19.121 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T14:32:19.121 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T14:32:19.122 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T14:32:19.122 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T14:32:19.122 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T14:32:19.122 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T14:32:19.122 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T14:32:19.123 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T14:32:19.123 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T14:32:19.123 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T14:32:19.123 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T14:32:19.124 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T14:32:19.124 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2022-11-27T14:32:19.124 INFO:teuthology.orchestra.run.smithi099.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T14:32:19.124 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T14:32:19.124 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T14:32:19.125 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T14:32:19.125 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T14:32:19.125 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2022-11-27T14:32:19.125 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T14:32:19.126 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T14:32:19.126 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T14:32:19.127 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T14:32:19.127 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T14:32:19.127 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T14:32:19.127 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T14:32:19.128 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T14:32:19.128 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T14:32:19.128 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T14:32:19.128 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T14:32:19.129 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T14:32:19.129 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T14:32:19.129 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T14:32:19.129 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T14:32:19.129 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T14:32:19.130 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T14:32:19.130 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T14:32:19.130 INFO:teuthology.orchestra.run.smithi099.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T14:32:19.130 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T14:32:19.131 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T14:32:19.131 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T14:32:19.131 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T14:32:19.132 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T14:32:19.132 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T14:32:19.132 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T14:32:19.132 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T14:32:19.132 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T14:32:19.133 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T14:32:19.133 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T14:32:19.133 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T14:32:19.133 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T14:32:19.134 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T14:32:19.134 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T14:32:19.134 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T14:32:19.134 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T14:32:19.134 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T14:32:19.135 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T14:32:19.135 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T14:32:19.135 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T14:32:19.136 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T14:32:19.136 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T14:32:19.136 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T14:32:19.136 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T14:32:19.136 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T14:32:19.137 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T14:32:19.137 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T14:32:19.137 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T14:32:19.137 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T14:32:19.138 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T14:32:19.138 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T14:32:19.138 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T14:32:19.138 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T14:32:19.138 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T14:32:19.139 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T14:32:19.139 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T14:32:19.139 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T14:32:19.139 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T14:32:19.140 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T14:32:19.140 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T14:32:19.140 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T14:32:19.140 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T14:32:19.140 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T14:32:19.141 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T14:32:19.141 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T14:32:19.141 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T14:32:19.141 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T14:32:19.142 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T14:32:19.142 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T14:32:19.142 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T14:32:19.143 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T14:32:19.143 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T14:32:19.143 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T14:32:19.144 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T14:32:19.144 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T14:32:19.144 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T14:32:19.144 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T14:32:19.144 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T14:32:19.145 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T14:32:19.145 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T14:32:19.145 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T14:32:19.145 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T14:32:19.145 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T14:32:19.146 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T14:32:19.146 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T14:32:19.146 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T14:32:19.147 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T14:32:19.147 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T14:32:19.147 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T14:32:19.147 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T14:32:19.148 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T14:32:19.148 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2022-11-27T14:32:19.148 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql 10 2022-11-27T14:32:19.148 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:32:19.148 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2022-11-27T14:32:19.149 INFO:teuthology.orchestra.run.smithi099.stdout:=========================================================================================================================== 2022-11-27T14:32:19.149 INFO:teuthology.orchestra.run.smithi099.stdout:Install 128 Packages 2022-11-27T14:32:19.149 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 5 Packages 2022-11-27T14:32:19.149 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:32:19.150 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 219 M 2022-11-27T14:32:19.151 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2022-11-27T14:32:19.355 INFO:teuthology.orchestra.run.smithi137.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 22 kB/s | 6.4 kB 00:00 2022-11-27T14:32:19.447 INFO:teuthology.orchestra.run.smithi099.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 20 kB/s | 6.4 kB 00:00 2022-11-27T14:32:20.606 INFO:teuthology.orchestra.run.smithi137.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 676 kB/s | 845 kB 00:01 2022-11-27T14:32:20.676 INFO:teuthology.orchestra.run.smithi099.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 688 kB/s | 845 kB 00:01 2022-11-27T14:32:20.856 INFO:teuthology.orchestra.run.smithi137.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 662 kB/s | 165 kB 00:00 2022-11-27T14:32:20.926 INFO:teuthology.orchestra.run.smithi099.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 661 kB/s | 165 kB 00:00 2022-11-27T14:32:21.898 INFO:teuthology.orchestra.run.smithi137.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 1.8 MB/s | 5.2 MB 00:02 2022-11-27T14:32:22.598 INFO:teuthology.orchestra.run.smithi137.stdout:(5/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 2.2 MB/s | 1.5 MB 00:00 2022-11-27T14:32:23.343 INFO:teuthology.orchestra.run.smithi099.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 1.2 MB/s | 5.2 MB 00:04 2022-11-27T14:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout:(6/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 769 kB/s | 2.3 MB 00:03 2022-11-27T14:32:23.884 INFO:teuthology.orchestra.run.smithi099.stdout:(5/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 785 kB/s | 2.3 MB 00:02 2022-11-27T14:32:23.998 INFO:teuthology.orchestra.run.smithi137.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 2.8 MB/s | 3.9 MB 00:01 2022-11-27T14:32:24.326 INFO:teuthology.orchestra.run.smithi099.stdout:(6/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 1.6 MB/s | 1.5 MB 00:00 2022-11-27T14:32:27.451 INFO:teuthology.orchestra.run.smithi099.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 1.1 MB/s | 3.9 MB 00:03 2022-11-27T14:32:30.582 INFO:teuthology.orchestra.run.smithi137.stdout:(8/133): ceph-radosgw-18.0.0-594.g697b0d99.el8. 2.8 MB/s | 18 MB 00:06 2022-11-27T14:32:30.821 INFO:teuthology.orchestra.run.smithi137.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 101 kB/s | 24 kB 00:00 2022-11-27T14:32:32.707 INFO:teuthology.orchestra.run.smithi137.stdout:(10/133): ceph-common-18.0.0-594.g697b0d99.el8. 1.8 MB/s | 24 MB 00:13 2022-11-27T14:32:32.882 INFO:teuthology.orchestra.run.smithi137.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 1.4 MB/s | 257 kB 00:00 2022-11-27T14:32:32.982 INFO:teuthology.orchestra.run.smithi137.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 305 kB/s | 30 kB 00:00 2022-11-27T14:32:33.249 INFO:teuthology.orchestra.run.smithi137.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 2.8 MB/s | 767 kB 00:00 2022-11-27T14:32:33.349 INFO:teuthology.orchestra.run.smithi137.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 1.3 MB/s | 137 kB 00:00 2022-11-27T14:32:33.458 INFO:teuthology.orchestra.run.smithi137.stdout:(15/133): librados-devel-18.0.0-594.g697b0d99.e 1.3 MB/s | 139 kB 00:00 2022-11-27T14:32:33.708 INFO:teuthology.orchestra.run.smithi137.stdout:(16/133): libradosstriper1-18.0.0-594.g697b0d99 1.9 MB/s | 474 kB 00:00 2022-11-27T14:32:33.842 INFO:teuthology.orchestra.run.smithi099.stdout:(8/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86_ 1.8 MB/s | 17 MB 00:09 2022-11-27T14:32:33.925 INFO:teuthology.orchestra.run.smithi099.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 293 kB/s | 24 kB 00:00 2022-11-27T14:32:35.411 INFO:teuthology.orchestra.run.smithi137.stdout:(17/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 3.8 MB/s | 6.5 MB 00:01 2022-11-27T14:32:35.508 INFO:teuthology.orchestra.run.smithi137.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 461 kB/s | 45 kB 00:00 2022-11-27T14:32:35.601 INFO:teuthology.orchestra.run.smithi137.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 1.2 MB/s | 109 kB 00:00 2022-11-27T14:32:35.701 INFO:teuthology.orchestra.run.smithi137.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 1.9 MB/s | 190 kB 00:00 2022-11-27T14:32:35.776 INFO:teuthology.orchestra.run.smithi099.stdout:(10/133): ceph-common-18.0.0-594.g697b0d99.el8. 1.5 MB/s | 24 MB 00:16 2022-11-27T14:32:36.009 INFO:teuthology.orchestra.run.smithi137.stdout:(21/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 1.4 MB/s | 17 MB 00:12 2022-11-27T14:32:36.035 INFO:teuthology.orchestra.run.smithi137.stdout:(22/133): python3-rados-18.0.0-594.g697b0d99.el 1.1 MB/s | 371 kB 00:00 2022-11-27T14:32:36.118 INFO:teuthology.orchestra.run.smithi137.stdout:(23/133): python3-rgw-18.0.0-594.g697b0d99.el8. 1.2 MB/s | 107 kB 00:00 2022-11-27T14:32:36.176 INFO:teuthology.orchestra.run.smithi099.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 644 kB/s | 257 kB 00:00 2022-11-27T14:32:36.202 INFO:teuthology.orchestra.run.smithi137.stdout:(24/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.1 MB/s | 94 kB 00:00 2022-11-27T14:32:36.259 INFO:teuthology.orchestra.run.smithi099.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 366 kB/s | 30 kB 00:00 2022-11-27T14:32:36.260 INFO:teuthology.orchestra.run.smithi137.stdout:(25/133): python3-rbd-18.0.0-594.g697b0d99.el8. 1.4 MB/s | 357 kB 00:00 2022-11-27T14:32:36.636 INFO:teuthology.orchestra.run.smithi137.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 466 kB/s | 175 kB 00:00 2022-11-27T14:32:36.710 INFO:teuthology.orchestra.run.smithi099.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 1.7 MB/s | 767 kB 00:00 2022-11-27T14:32:36.736 INFO:teuthology.orchestra.run.smithi137.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 229 kB/s | 23 kB 00:00 2022-11-27T14:32:36.810 INFO:teuthology.orchestra.run.smithi099.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 1.3 MB/s | 137 kB 00:00 2022-11-27T14:32:36.910 INFO:teuthology.orchestra.run.smithi099.stdout:(15/133): librados-devel-18.0.0-594.g697b0d99.e 1.4 MB/s | 139 kB 00:00 2022-11-27T14:32:36.920 INFO:teuthology.orchestra.run.smithi137.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 644 kB/s | 118 kB 00:00 2022-11-27T14:32:37.128 INFO:teuthology.orchestra.run.smithi137.stdout:(29/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 3.7 MB/s | 3.5 MB 00:00 2022-11-27T14:32:37.160 INFO:teuthology.orchestra.run.smithi099.stdout:(16/133): libradosstriper1-18.0.0-594.g697b0d99 1.9 MB/s | 474 kB 00:00 2022-11-27T14:32:38.120 INFO:teuthology.orchestra.run.smithi137.stdout:(30/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 1.4 MB/s | 1.7 MB 00:01 2022-11-27T14:32:38.354 INFO:teuthology.orchestra.run.smithi137.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 1.0 MB/s | 234 kB 00:00 2022-11-27T14:32:38.463 INFO:teuthology.orchestra.run.smithi137.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 434 kB/s | 47 kB 00:00 2022-11-27T14:32:38.571 INFO:teuthology.orchestra.run.smithi137.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 133 kB/s | 14 kB 00:00 2022-11-27T14:32:38.863 INFO:teuthology.orchestra.run.smithi137.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 901 kB/s | 263 kB 00:00 2022-11-27T14:32:39.047 INFO:teuthology.orchestra.run.smithi137.stdout:(35/133): ceph-mgr-diskprediction-local-18.0.0- 3.9 MB/s | 7.4 MB 00:01 2022-11-27T14:32:39.080 INFO:teuthology.orchestra.run.smithi137.stdout:(36/133): cephadm-18.0.0-594.g697b0d99.el8.noar 375 kB/s | 81 kB 00:00 2022-11-27T14:32:39.131 INFO:teuthology.orchestra.run.smithi137.stdout:(37/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 185 kB/s | 15 kB 00:00 2022-11-27T14:32:39.323 INFO:teuthology.orchestra.run.smithi137.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 307 kB/s | 59 kB 00:00 2022-11-27T14:32:39.348 INFO:teuthology.orchestra.run.smithi137.stdout:(39/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2022-11-27T14:32:39.352 INFO:teuthology.orchestra.run.smithi099.stdout:(17/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 1.5 MB/s | 18 MB 00:11 2022-11-27T14:32:39.407 INFO:teuthology.orchestra.run.smithi137.stdout:(40/133): libunwind-1.3.1-3.el8.x86_64.rpm 898 kB/s | 75 kB 00:00 2022-11-27T14:32:39.432 INFO:teuthology.orchestra.run.smithi137.stdout:(41/133): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2022-11-27T14:32:39.436 INFO:teuthology.orchestra.run.smithi099.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 538 kB/s | 45 kB 00:00 2022-11-27T14:32:39.457 INFO:teuthology.orchestra.run.smithi137.stdout:(42/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 869 kB/s | 44 kB 00:00 2022-11-27T14:32:39.483 INFO:teuthology.orchestra.run.smithi137.stdout:(43/133): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2022-11-27T14:32:39.508 INFO:teuthology.orchestra.run.smithi137.stdout:(44/133): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2022-11-27T14:32:39.566 INFO:teuthology.orchestra.run.smithi137.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2022-11-27T14:32:39.577 INFO:teuthology.orchestra.run.smithi099.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 770 kB/s | 109 kB 00:00 2022-11-27T14:32:39.658 INFO:teuthology.orchestra.run.smithi137.stdout:(46/133): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2022-11-27T14:32:39.683 INFO:teuthology.orchestra.run.smithi137.stdout:(47/133): python3-cheroot-8.5.2-1.el8.noarch.rp 983 kB/s | 173 kB 00:00 2022-11-27T14:32:39.709 INFO:teuthology.orchestra.run.smithi137.stdout:(48/133): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2022-11-27T14:32:39.728 INFO:teuthology.orchestra.run.smithi099.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 1.2 MB/s | 190 kB 00:00 2022-11-27T14:32:39.734 INFO:teuthology.orchestra.run.smithi137.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2022-11-27T14:32:39.751 INFO:teuthology.orchestra.run.smithi137.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 420 kB/s | 18 kB 00:00 2022-11-27T14:32:39.810 INFO:teuthology.orchestra.run.smithi137.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 837 kB/s | 49 kB 00:00 2022-11-27T14:32:39.860 INFO:teuthology.orchestra.run.smithi137.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2022-11-27T14:32:39.902 INFO:teuthology.orchestra.run.smithi137.stdout:(53/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2022-11-27T14:32:39.945 INFO:teuthology.orchestra.run.smithi099.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 1.7 MB/s | 371 kB 00:00 2022-11-27T14:32:39.961 INFO:teuthology.orchestra.run.smithi137.stdout:(54/133): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2022-11-27T14:32:40.003 INFO:teuthology.orchestra.run.smithi137.stdout:(55/133): python3-portend-2.6-1.el8.noarch.rpm 375 kB/s | 16 kB 00:00 2022-11-27T14:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout:(56/133): python3-repoze-lru-0.7-6.el8.noarch.r 677 kB/s | 34 kB 00:00 2022-11-27T14:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout:(57/133): python3-routes-2.4.1-12.el8.noarch.rp 3.9 MB/s | 196 kB 00:00 2022-11-27T14:32:40.145 INFO:teuthology.orchestra.run.smithi099.stdout:(22/133): python3-rbd-18.0.0-594.g697b0d99.el8. 1.7 MB/s | 357 kB 00:00 2022-11-27T14:32:40.146 INFO:teuthology.orchestra.run.smithi137.stdout:(58/133): python3-rsa-4.8-1.el8.noarch.rpm 1.4 MB/s | 61 kB 00:00 2022-11-27T14:32:40.187 INFO:teuthology.orchestra.run.smithi137.stdout:(59/133): python3-simplegeneric-0.8.1-17.el8.no 451 kB/s | 19 kB 00:00 2022-11-27T14:32:40.229 INFO:teuthology.orchestra.run.smithi137.stdout:(60/133): python3-singledispatch-3.4.0.3-18.el8 577 kB/s | 24 kB 00:00 2022-11-27T14:32:40.237 INFO:teuthology.orchestra.run.smithi099.stdout:(23/133): python3-rgw-18.0.0-594.g697b0d99.el8. 1.1 MB/s | 107 kB 00:00 2022-11-27T14:32:40.304 INFO:teuthology.orchestra.run.smithi137.stdout:(61/133): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2022-11-27T14:32:40.355 INFO:teuthology.orchestra.run.smithi137.stdout:(62/133): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2022-11-27T14:32:40.379 INFO:teuthology.orchestra.run.smithi099.stdout:(24/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 2.0 MB/s | 6.5 MB 00:03 2022-11-27T14:32:40.388 INFO:teuthology.orchestra.run.smithi137.stdout:(63/133): python3-kubernetes-11.0.0-6.el8.noarc 1.8 MB/s | 1.2 MB 00:00 2022-11-27T14:32:40.405 INFO:teuthology.orchestra.run.smithi099.stdout:(25/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 566 kB/s | 94 kB 00:00 2022-11-27T14:32:40.414 INFO:teuthology.orchestra.run.smithi137.stdout:(64/133): python3-waitress-1.2.1-2.el8.1.noarch 4.0 MB/s | 241 kB 00:00 2022-11-27T14:32:40.464 INFO:teuthology.orchestra.run.smithi137.stdout:(65/133): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2022-11-27T14:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout:(66/133): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2022-11-27T14:32:40.546 INFO:teuthology.orchestra.run.smithi099.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 1.2 MB/s | 175 kB 00:00 2022-11-27T14:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout:(67/133): python3-webob-1.8.5-1.el8.1.noarch.rp 1.4 MB/s | 251 kB 00:00 2022-11-27T14:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 431 kB/s | 23 kB 00:00 2022-11-27T14:32:40.622 INFO:teuthology.orchestra.run.smithi099.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 304 kB/s | 23 kB 00:00 2022-11-27T14:32:40.632 INFO:teuthology.orchestra.run.smithi137.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2022-11-27T14:32:40.705 INFO:teuthology.orchestra.run.smithi099.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.4 MB/s | 118 kB 00:00 2022-11-27T14:32:41.157 INFO:teuthology.orchestra.run.smithi137.stdout:(70/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 5.3 MB/s | 55 MB 00:10 2022-11-27T14:32:41.182 INFO:teuthology.orchestra.run.smithi137.stdout:(71/133): postgresql-contrib-10.21-2.module+el8 1.4 MB/s | 808 kB 00:00 2022-11-27T14:32:41.241 INFO:teuthology.orchestra.run.smithi137.stdout:(72/133): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2022-11-27T14:32:41.247 INFO:teuthology.orchestra.run.smithi099.stdout:(29/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 3.0 MB/s | 1.7 MB 00:00 2022-11-27T14:32:41.283 INFO:teuthology.orchestra.run.smithi137.stdout:(73/133): postgresql-10.21-2.module+el8.6.0+153 15 MB/s | 1.5 MB 00:00 2022-11-27T14:32:41.380 INFO:teuthology.orchestra.run.smithi137.stdout:(74/133): python3-numpy-1.14.3-10.el8.x86_64.rp 27 MB/s | 3.7 MB 00:00 2022-11-27T14:32:41.425 INFO:teuthology.orchestra.run.smithi137.stdout:(75/133): python3-werkzeug-0.12.2-4.el8.noarch. 10 MB/s | 457 kB 00:00 2022-11-27T14:32:41.500 INFO:teuthology.orchestra.run.smithi137.stdout:(76/133): postgresql-server-10.21-2.module+el8. 23 MB/s | 5.1 MB 00:00 2022-11-27T14:32:41.534 INFO:teuthology.orchestra.run.smithi137.stdout:(77/133): python3-markupsafe-0.23-19.el8.x86_64 357 kB/s | 39 kB 00:00 2022-11-27T14:32:41.559 INFO:teuthology.orchestra.run.smithi137.stdout:(78/133): python3-prettytable-0.7.2-14.el8.noar 767 kB/s | 44 kB 00:00 2022-11-27T14:32:41.601 INFO:teuthology.orchestra.run.smithi137.stdout:(79/133): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2022-11-27T14:32:41.635 INFO:teuthology.orchestra.run.smithi099.stdout:(30/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 2.8 MB/s | 3.5 MB 00:01 2022-11-27T14:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout:(80/133): python3-jinja2-2.10.1-3.el8.noarch.rp 5.7 MB/s | 538 kB 00:00 2022-11-27T14:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout:(81/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.4 MB/s | 225 kB 00:00 2022-11-27T14:32:41.727 INFO:teuthology.orchestra.run.smithi137.stdout:(82/133): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2022-11-27T14:32:41.761 INFO:teuthology.orchestra.run.smithi137.stdout:(83/133): thrift-0.13.0-2.el8.x86_64.rpm 1.5 MB/s | 1.7 MB 00:01 2022-11-27T14:32:41.765 INFO:teuthology.orchestra.run.smithi099.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 1.8 MB/s | 234 kB 00:00 2022-11-27T14:32:41.794 INFO:teuthology.orchestra.run.smithi137.stdout:(84/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2022-11-27T14:32:41.836 INFO:teuthology.orchestra.run.smithi137.stdout:(85/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2022-11-27T14:32:41.840 INFO:teuthology.orchestra.run.smithi099.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 627 kB/s | 47 kB 00:00 2022-11-27T14:32:41.912 INFO:teuthology.orchestra.run.smithi137.stdout:(86/133): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2022-11-27T14:32:41.947 INFO:teuthology.orchestra.run.smithi099.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 135 kB/s | 14 kB 00:00 2022-11-27T14:32:42.082 INFO:teuthology.orchestra.run.smithi099.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 1.9 MB/s | 263 kB 00:00 2022-11-27T14:32:42.095 INFO:teuthology.orchestra.run.smithi137.stdout:(87/133): openblas-threads-0.3.15-3.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2022-11-27T14:32:42.157 INFO:teuthology.orchestra.run.smithi099.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 1.1 MB/s | 81 kB 00:00 2022-11-27T14:32:42.187 INFO:teuthology.orchestra.run.smithi137.stdout:(88/133): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2022-11-27T14:32:42.213 INFO:teuthology.orchestra.run.smithi137.stdout:(89/133): flex-2.6.1-9.el8.x86_64.rpm 1.0 MB/s | 320 kB 00:00 2022-11-27T14:32:42.250 INFO:teuthology.orchestra.run.smithi099.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 168 kB/s | 15 kB 00:00 2022-11-27T14:32:42.254 INFO:teuthology.orchestra.run.smithi137.stdout:(90/133): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2022-11-27T14:32:42.305 INFO:teuthology.orchestra.run.smithi137.stdout:(91/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-11-27T14:32:42.338 INFO:teuthology.orchestra.run.smithi137.stdout:(92/133): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2022-11-27T14:32:42.431 INFO:teuthology.orchestra.run.smithi137.stdout:(93/133): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2022-11-27T14:32:42.450 INFO:teuthology.orchestra.run.smithi099.stdout:(37/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.5 MB/s | 306 kB 00:00 2022-11-27T14:32:42.472 INFO:teuthology.orchestra.run.smithi137.stdout:(94/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 946 kB/s | 126 kB 00:00 2022-11-27T14:32:42.498 INFO:teuthology.orchestra.run.smithi137.stdout:(95/133): python3-jsonpointer-1.10-11.el8.noarc 299 kB/s | 20 kB 00:00 2022-11-27T14:32:42.529 INFO:teuthology.orchestra.run.smithi099.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 752 kB/s | 59 kB 00:00 2022-11-27T14:32:42.548 INFO:teuthology.orchestra.run.smithi137.stdout:(96/133): libpq-13.5-1.el8.x86_64.rpm 2.6 MB/s | 198 kB 00:00 2022-11-27T14:32:42.573 INFO:teuthology.orchestra.run.smithi137.stdout:(97/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2022-11-27T14:32:42.584 INFO:teuthology.orchestra.run.smithi099.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2022-11-27T14:32:42.638 INFO:teuthology.orchestra.run.smithi137.stdout:(98/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2022-11-27T14:32:42.666 INFO:teuthology.orchestra.run.smithi137.stdout:(99/133): python3-jsonpatch-1.21-2.el8.noarch.r 292 kB/s | 27 kB 00:00 2022-11-27T14:32:42.667 INFO:teuthology.orchestra.run.smithi099.stdout:(40/133): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2022-11-27T14:32:42.716 INFO:teuthology.orchestra.run.smithi137.stdout:(100/133): python3-pyasn1-modules-0.3.7-6.el8.n 2.2 MB/s | 110 kB 00:00 2022-11-27T14:32:42.717 INFO:teuthology.orchestra.run.smithi099.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 874 kB/s | 44 kB 00:00 2022-11-27T14:32:42.741 INFO:teuthology.orchestra.run.smithi137.stdout:(101/133): uuid-1.6.2-43.el8.x86_64.rpm 625 kB/s | 64 kB 00:00 2022-11-27T14:32:42.768 INFO:teuthology.orchestra.run.smithi099.stdout:(42/133): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2022-11-27T14:32:42.843 INFO:teuthology.orchestra.run.smithi099.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 440 kB/s | 33 kB 00:00 2022-11-27T14:32:42.893 INFO:teuthology.orchestra.run.smithi099.stdout:(44/133): python3-certifi-2018.10.15-7.el8.noar 327 kB/s | 16 kB 00:00 2022-11-27T14:32:42.908 INFO:teuthology.orchestra.run.smithi137.stdout:(102/133): python3-scipy-1.0.0-21.module+el8.5. 20 MB/s | 14 MB 00:00 2022-11-27T14:32:42.944 INFO:teuthology.orchestra.run.smithi099.stdout:(45/133): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2022-11-27T14:32:42.950 INFO:teuthology.orchestra.run.smithi137.stdout:(103/133): pcre2-devel-10.32-2.el8.x86_64.rpm 2.5 MB/s | 605 kB 00:00 2022-11-27T14:32:42.976 INFO:teuthology.orchestra.run.smithi137.stdout:(104/133): fuse-2.9.7-15.el8.x86_64.rpm 354 kB/s | 83 kB 00:00 2022-11-27T14:32:43.000 INFO:teuthology.orchestra.run.smithi137.stdout:(105/133): libsepol-devel-2.9-3.el8.x86_64.rpm 970 kB/s | 87 kB 00:00 2022-11-27T14:32:43.034 INFO:teuthology.orchestra.run.smithi137.stdout:(106/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2022-11-27T14:32:43.059 INFO:teuthology.orchestra.run.smithi137.stdout:(107/133): libverto-devel-0.3.0-5.el8.x86_64.rp 311 kB/s | 18 kB 00:00 2022-11-27T14:32:43.085 INFO:teuthology.orchestra.run.smithi137.stdout:(108/133): libselinux-devel-2.9-5.el8.x86_64.rp 1.8 MB/s | 200 kB 00:00 2022-11-27T14:32:43.109 INFO:teuthology.orchestra.run.smithi099.stdout:(46/133): ceph-mgr-diskprediction-local-18.0.0- 4.0 MB/s | 7.4 MB 00:01 2022-11-27T14:32:43.143 INFO:teuthology.orchestra.run.smithi137.stdout:(109/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 430 kB/s | 47 kB 00:00 2022-11-27T14:32:43.145 INFO:teuthology.orchestra.run.smithi099.stdout:(47/133): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2022-11-27T14:32:43.169 INFO:teuthology.orchestra.run.smithi137.stdout:(110/133): python3-oauthlib-2.1.0-1.el8.noarch. 1.9 MB/s | 155 kB 00:00 2022-11-27T14:32:43.195 INFO:teuthology.orchestra.run.smithi099.stdout:(48/133): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2022-11-27T14:32:43.202 INFO:teuthology.orchestra.run.smithi137.stdout:(111/133): python3-cryptography-3.2.1-5.el8.x86 3.8 MB/s | 559 kB 00:00 2022-11-27T14:32:43.227 INFO:teuthology.orchestra.run.smithi137.stdout:(112/133): python3-pycparser-2.14-14.el8.noarch 1.3 MB/s | 109 kB 00:00 2022-11-27T14:32:43.245 INFO:teuthology.orchestra.run.smithi099.stdout:(49/133): python3-jaraco-functools-2.0-4.el8.no 352 kB/s | 18 kB 00:00 2022-11-27T14:32:43.269 INFO:teuthology.orchestra.run.smithi137.stdout:(113/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.5 MB/s | 560 kB 00:00 2022-11-27T14:32:43.271 INFO:teuthology.orchestra.run.smithi099.stdout:(50/133): python3-google-auth-1.1.1-10.el8.noar 503 kB/s | 81 kB 00:00 2022-11-27T14:32:43.295 INFO:teuthology.orchestra.run.smithi137.stdout:(114/133): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.9 MB/s | 171 kB 00:00 2022-11-27T14:32:43.321 INFO:teuthology.orchestra.run.smithi099.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 974 kB/s | 49 kB 00:00 2022-11-27T14:32:43.328 INFO:teuthology.orchestra.run.smithi137.stdout:(115/133): python3-jwt-1.6.1-2.el8.noarch.rpm 431 kB/s | 43 kB 00:00 2022-11-27T14:32:43.374 INFO:teuthology.orchestra.run.smithi137.stdout:(116/133): zlib-devel-1.2.11-18.el8_5.x86_64.rp 550 kB/s | 58 kB 00:00 2022-11-27T14:32:43.403 INFO:teuthology.orchestra.run.smithi099.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 716 kB/s | 59 kB 00:00 2022-11-27T14:32:43.404 INFO:teuthology.orchestra.run.smithi137.stdout:(117/133): elfutils-libelf-devel-0.186-1.el8.x8 561 kB/s | 60 kB 00:00 2022-11-27T14:32:43.437 INFO:teuthology.orchestra.run.smithi137.stdout:(118/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.2 MB/s | 238 kB 00:00 2022-11-27T14:32:43.454 INFO:teuthology.orchestra.run.smithi137.stdout:(119/133): libaio-devel-0.3.112-1.el8.x86_64.rp 235 kB/s | 19 kB 00:00 2022-11-27T14:32:43.463 INFO:teuthology.orchestra.run.smithi099.stdout:(53/133): python3-kubernetes-11.0.0-6.el8.noarc 5.4 MB/s | 1.2 MB 00:00 2022-11-27T14:32:43.480 INFO:teuthology.orchestra.run.smithi137.stdout:(120/133): libuuid-devel-2.32.1-35.el8.x86_64.r 1.3 MB/s | 98 kB 00:00 2022-11-27T14:32:43.488 INFO:teuthology.orchestra.run.smithi099.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 711 kB/s | 60 kB 00:00 2022-11-27T14:32:43.522 INFO:teuthology.orchestra.run.smithi137.stdout:(121/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2022-11-27T14:32:43.523 INFO:teuthology.orchestra.run.smithi099.stdout:(55/133): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2022-11-27T14:32:43.539 INFO:teuthology.orchestra.run.smithi099.stdout:(56/133): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2022-11-27T14:32:43.547 INFO:teuthology.orchestra.run.smithi137.stdout:(122/133): python3-requests-oauthlib-1.0.0-1.el 463 kB/s | 43 kB 00:00 2022-11-27T14:32:43.572 INFO:teuthology.orchestra.run.smithi137.stdout:(123/133): xfsprogs-devel-5.0.0-10.el8.x86_64.r 793 kB/s | 72 kB 00:00 2022-11-27T14:32:43.573 INFO:teuthology.orchestra.run.smithi099.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 669 kB/s | 34 kB 00:00 2022-11-27T14:32:43.606 INFO:teuthology.orchestra.run.smithi137.stdout:(124/133): libcom_err-devel-1.45.6-4.el8.x86_64 461 kB/s | 39 kB 00:00 2022-11-27T14:32:43.623 INFO:teuthology.orchestra.run.smithi099.stdout:(58/133): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2022-11-27T14:32:43.649 INFO:teuthology.orchestra.run.smithi099.stdout:(59/133): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2022-11-27T14:32:43.656 INFO:teuthology.orchestra.run.smithi137.stdout:(125/133): keyutils-libs-devel-1.5.10-9.el8.x86 583 kB/s | 48 kB 00:00 2022-11-27T14:32:43.698 INFO:teuthology.orchestra.run.smithi099.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 252 kB/s | 19 kB 00:00 2022-11-27T14:32:43.698 INFO:teuthology.orchestra.run.smithi137.stdout:(126/133): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2022-11-27T14:32:43.724 INFO:teuthology.orchestra.run.smithi099.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 318 kB/s | 24 kB 00:00 2022-11-27T14:32:43.741 INFO:teuthology.orchestra.run.smithi099.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 682 kB/s | 29 kB 00:00 2022-11-27T14:32:43.748 INFO:teuthology.orchestra.run.smithi137.stdout:(127/133): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2022-11-27T14:32:43.766 INFO:teuthology.orchestra.run.smithi099.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 659 kB/s | 27 kB 00:00 2022-11-27T14:32:43.774 INFO:teuthology.orchestra.run.smithi137.stdout:(128/133): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.8 MB/s | 644 kB 00:00 2022-11-27T14:32:43.800 INFO:teuthology.orchestra.run.smithi099.stdout:(64/133): python3-waitress-1.2.1-2.el8.1.noarch 4.0 MB/s | 241 kB 00:00 2022-11-27T14:32:43.816 INFO:teuthology.orchestra.run.smithi137.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 17 MB/s | 709 kB 00:00 2022-11-27T14:32:43.825 INFO:teuthology.orchestra.run.smithi099.stdout:(65/133): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2022-11-27T14:32:43.842 INFO:teuthology.orchestra.run.smithi099.stdout:(66/133): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2022-11-27T14:32:43.876 INFO:teuthology.orchestra.run.smithi099.stdout:(67/133): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2022-11-27T14:32:43.893 INFO:teuthology.orchestra.run.smithi099.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2022-11-27T14:32:43.898 INFO:teuthology.orchestra.run.smithi137.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 18 MB/s | 1.5 MB 00:00 2022-11-27T14:32:43.918 INFO:teuthology.orchestra.run.smithi099.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2022-11-27T14:32:43.950 INFO:teuthology.orchestra.run.smithi137.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 15 MB/s | 798 kB 00:00 2022-11-27T14:32:44.000 INFO:teuthology.orchestra.run.smithi099.stdout:(70/133): postgresql-contrib-10.21-2.module+el8 9.7 MB/s | 808 kB 00:00 2022-11-27T14:32:44.052 INFO:teuthology.orchestra.run.smithi099.stdout:(71/133): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2022-11-27T14:32:44.111 INFO:teuthology.orchestra.run.smithi099.stdout:(72/133): postgresql-10.21-2.module+el8.6.0+153 26 MB/s | 1.5 MB 00:00 2022-11-27T14:32:44.153 INFO:teuthology.orchestra.run.smithi099.stdout:(73/133): thrift-0.13.0-2.el8.x86_64.rpm 6.3 MB/s | 1.7 MB 00:00 2022-11-27T14:32:44.246 INFO:teuthology.orchestra.run.smithi099.stdout:(74/133): python3-numpy-1.14.3-10.el8.x86_64.rp 28 MB/s | 3.7 MB 00:00 2022-11-27T14:32:44.337 INFO:teuthology.orchestra.run.smithi099.stdout:(75/133): postgresql-server-10.21-2.module+el8. 28 MB/s | 5.1 MB 00:00 2022-11-27T14:32:44.362 INFO:teuthology.orchestra.run.smithi099.stdout:(76/133): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2022-11-27T14:32:44.396 INFO:teuthology.orchestra.run.smithi099.stdout:(77/133): python3-markupsafe-0.23-19.el8.x86_64 665 kB/s | 39 kB 00:00 2022-11-27T14:32:44.421 INFO:teuthology.orchestra.run.smithi099.stdout:(78/133): python3-prettytable-0.7.2-14.el8.noar 750 kB/s | 44 kB 00:00 2022-11-27T14:32:44.463 INFO:teuthology.orchestra.run.smithi099.stdout:(79/133): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2022-11-27T14:32:44.489 INFO:teuthology.orchestra.run.smithi099.stdout:(80/133): python3-jinja2-2.10.1-3.el8.noarch.rp 7.8 MB/s | 538 kB 00:00 2022-11-27T14:32:44.539 INFO:teuthology.orchestra.run.smithi099.stdout:(81/133): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2022-11-27T14:32:44.597 INFO:teuthology.orchestra.run.smithi099.stdout:(82/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.0 MB/s | 225 kB 00:00 2022-11-27T14:32:44.640 INFO:teuthology.orchestra.run.smithi099.stdout:(83/133): python3-mako-1.0.6-13.el8.noarch.rpm 3.8 MB/s | 157 kB 00:00 2022-11-27T14:32:44.732 INFO:teuthology.orchestra.run.smithi099.stdout:(84/133): openblas-threads-0.3.15-3.el8.x86_64. 25 MB/s | 4.9 MB 00:00 2022-11-27T14:32:44.741 INFO:teuthology.orchestra.run.smithi137.stdout:(132/133): librbd1-18.0.0-594.g697b0d99.el8.x86 3.7 MB/s | 3.6 MB 00:00 2022-11-27T14:32:44.773 INFO:teuthology.orchestra.run.smithi099.stdout:(85/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2022-11-27T14:32:44.867 INFO:teuthology.orchestra.run.smithi099.stdout:(86/133): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2022-11-27T14:32:44.907 INFO:teuthology.orchestra.run.smithi099.stdout:(87/133): socat-1.7.4.1-1.el8.x86_64.rpm 2.4 MB/s | 323 kB 00:00 2022-11-27T14:32:44.933 INFO:teuthology.orchestra.run.smithi099.stdout:(88/133): flex-2.6.1-9.el8.x86_64.rpm 4.8 MB/s | 320 kB 00:00 2022-11-27T14:32:44.975 INFO:teuthology.orchestra.run.smithi099.stdout:(89/133): jq-1.6-3.el8.x86_64.rpm 4.9 MB/s | 202 kB 00:00 2022-11-27T14:32:45.067 INFO:teuthology.orchestra.run.smithi099.stdout:(90/133): openblas-0.3.15-3.el8.x86_64.rpm 30 MB/s | 4.7 MB 00:00 2022-11-27T14:32:45.109 INFO:teuthology.orchestra.run.smithi099.stdout:(91/133): automake-1.16.1-7.el8.noarch.rpm 17 MB/s | 713 kB 00:00 2022-11-27T14:32:45.170 INFO:teuthology.orchestra.run.smithi099.stdout:(92/133): python3-pytz-2017.2-9.el8.noarch.rpm 893 kB/s | 54 kB 00:00 2022-11-27T14:32:45.218 INFO:teuthology.orchestra.run.smithi099.stdout:(93/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.6 MB/s | 126 kB 00:00 2022-11-27T14:32:45.260 INFO:teuthology.orchestra.run.smithi099.stdout:(94/133): python3-jsonpointer-1.10-11.el8.noarc 481 kB/s | 20 kB 00:00 2022-11-27T14:32:45.402 INFO:teuthology.orchestra.run.smithi099.stdout:(95/133): python3-scipy-1.0.0-21.module+el8.5.0 32 MB/s | 14 MB 00:00 2022-11-27T14:32:45.464 INFO:teuthology.orchestra.run.smithi099.stdout:(96/133): libpq-13.5-1.el8.x86_64.rpm 968 kB/s | 198 kB 00:00 2022-11-27T14:32:45.494 INFO:teuthology.orchestra.run.smithi099.stdout:(97/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2022-11-27T14:32:45.532 INFO:teuthology.orchestra.run.smithi099.stdout:(98/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2022-11-27T14:32:45.555 INFO:teuthology.orchestra.run.smithi099.stdout:(99/133): python3-jsonpatch-1.21-2.el8.noarch.r 444 kB/s | 27 kB 00:00 2022-11-27T14:32:45.588 INFO:teuthology.orchestra.run.smithi099.stdout:(100/133): uuid-1.6.2-43.el8.x86_64.rpm 1.0 MB/s | 64 kB 00:00 2022-11-27T14:32:45.600 INFO:teuthology.orchestra.run.smithi137.stdout:(133/133): librados2-18.0.0-594.g697b0d99.el8.x 1.9 MB/s | 3.6 MB 00:01 2022-11-27T14:32:45.603 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2022-11-27T14:32:45.603 INFO:teuthology.orchestra.run.smithi137.stdout:Total 8.3 MB/s | 219 MB 00:26 2022-11-27T14:32:45.605 INFO:teuthology.orchestra.run.smithi099.stdout:(101/133): python3-pyasn1-modules-0.3.7-6.el8.n 2.1 MB/s | 110 kB 00:00 2022-11-27T14:32:45.647 INFO:teuthology.orchestra.run.smithi099.stdout:(102/133): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2022-11-27T14:32:45.664 INFO:teuthology.orchestra.run.smithi099.stdout:(103/133): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2022-11-27T14:32:45.698 INFO:teuthology.orchestra.run.smithi099.stdout:(104/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2022-11-27T14:32:45.723 INFO:teuthology.orchestra.run.smithi099.stdout:(105/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2022-11-27T14:32:45.782 INFO:teuthology.orchestra.run.smithi099.stdout:(106/133): libselinux-devel-2.9-5.el8.x86_64.rp 2.3 MB/s | 200 kB 00:00 2022-11-27T14:32:45.807 INFO:teuthology.orchestra.run.smithi099.stdout:(107/133): libverto-devel-0.3.0-5.el8.x86_64.rp 214 kB/s | 18 kB 00:00 2022-11-27T14:32:45.849 INFO:teuthology.orchestra.run.smithi099.stdout:(108/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 699 kB/s | 47 kB 00:00 2022-11-27T14:32:45.891 INFO:teuthology.orchestra.run.smithi099.stdout:(109/133): python3-cryptography-3.2.1-5.el8.x86 6.5 MB/s | 559 kB 00:00 2022-11-27T14:32:45.925 INFO:teuthology.orchestra.run.smithi099.stdout:(110/133): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2022-11-27T14:32:45.950 INFO:teuthology.orchestra.run.smithi099.stdout:(111/133): python3-pycparser-2.14-14.el8.noarch 1.8 MB/s | 109 kB 00:00 2022-11-27T14:32:45.992 INFO:teuthology.orchestra.run.smithi099.stdout:(112/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2022-11-27T14:32:46.018 INFO:teuthology.orchestra.run.smithi099.stdout:(113/133): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2022-11-27T14:32:46.076 INFO:teuthology.orchestra.run.smithi099.stdout:(114/133): python3-jwt-1.6.1-2.el8.noarch.rpm 509 kB/s | 43 kB 00:00 2022-11-27T14:32:46.101 INFO:teuthology.orchestra.run.smithi099.stdout:(115/133): zlib-devel-1.2.11-18.el8_5.x86_64.rp 700 kB/s | 58 kB 00:00 2022-11-27T14:32:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:(116/133): elfutils-libelf-devel-0.186-1.el8.x8 895 kB/s | 60 kB 00:00 2022-11-27T14:32:46.169 INFO:teuthology.orchestra.run.smithi099.stdout:(117/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2022-11-27T14:32:46.203 INFO:teuthology.orchestra.run.smithi099.stdout:(118/133): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2022-11-27T14:32:46.228 INFO:teuthology.orchestra.run.smithi099.stdout:(119/133): libuuid-devel-2.32.1-35.el8.x86_64.r 1.7 MB/s | 98 kB 00:00 2022-11-27T14:32:46.261 INFO:teuthology.orchestra.run.smithi099.stdout:(120/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2022-11-27T14:32:46.287 INFO:teuthology.orchestra.run.smithi099.stdout:(121/133): python3-requests-oauthlib-1.0.0-1.el 743 kB/s | 43 kB 00:00 2022-11-27T14:32:46.349 INFO:teuthology.orchestra.run.smithi099.stdout:(122/133): xfsprogs-devel-5.0.0-10.el8.x86_64.r 827 kB/s | 72 kB 00:00 2022-11-27T14:32:46.379 INFO:teuthology.orchestra.run.smithi099.stdout:(123/133): libcom_err-devel-1.45.6-4.el8.x86_64 421 kB/s | 39 kB 00:00 2022-11-27T14:32:46.421 INFO:teuthology.orchestra.run.smithi099.stdout:(124/133): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2022-11-27T14:32:46.459 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2022-11-27T14:32:46.488 INFO:teuthology.orchestra.run.smithi099.stdout:(125/133): openssl-devel-1.1.1k-7.el8_6.x86_64. 17 MB/s | 2.3 MB 00:00 2022-11-27T14:32:46.530 INFO:teuthology.orchestra.run.smithi099.stdout:(126/133): libgfortran-8.5.0-10.1.el8_6.x86_64. 5.8 MB/s | 644 kB 00:00 2022-11-27T14:32:46.618 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2022-11-27T14:32:46.620 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2022-11-27T14:32:46.647 INFO:teuthology.orchestra.run.smithi099.stdout:(127/133): NetworkManager-initscripts-updown-1. 868 kB/s | 138 kB 00:00 2022-11-27T14:32:47.791 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2022-11-27T14:32:47.795 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2022-11-27T14:32:47.897 INFO:teuthology.orchestra.run.smithi099.stdout:(128/133): librados2-18.0.0-594.g697b0d99.el8.x 2.7 MB/s | 3.6 MB 00:01 2022-11-27T14:32:47.948 INFO:teuthology.orchestra.run.smithi099.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 14 MB/s | 709 kB 00:00 2022-11-27T14:32:48.006 INFO:teuthology.orchestra.run.smithi099.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 1.5 MB 00:00 2022-11-27T14:32:48.048 INFO:teuthology.orchestra.run.smithi099.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2022-11-27T14:32:48.940 INFO:teuthology.orchestra.run.smithi099.stdout:(132/133): librbd1-18.0.0-594.g697b0d99.el8.x86 1.6 MB/s | 3.6 MB 00:02 2022-11-27T14:32:50.048 INFO:teuthology.orchestra.run.smithi099.stdout:(133/133): ceph-test-18.0.0-594.g697b0d99.el8.x 3.4 MB/s | 55 MB 00:16 2022-11-27T14:32:50.052 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2022-11-27T14:32:50.052 INFO:teuthology.orchestra.run.smithi099.stdout:Total 7.1 MB/s | 219 MB 00:30 2022-11-27T14:32:50.897 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2022-11-27T14:32:51.056 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2022-11-27T14:32:51.056 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2022-11-27T14:32:52.209 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2022-11-27T14:32:52.214 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2022-11-27T14:32:53.068 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2022-11-27T14:32:53.327 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T14:32:53.547 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T14:32:53.775 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T14:32:53.932 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T14:32:54.110 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T14:32:54.168 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T14:32:54.385 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 5/138 2022-11-27T14:32:54.577 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T14:32:54.957 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T14:32:55.218 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T14:32:55.605 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T14:32:55.827 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T14:32:55.951 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T14:32:56.151 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T14:32:56.265 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T14:32:56.473 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T14:32:56.761 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T14:32:56.964 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T14:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T14:32:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2022-11-27T14:32:57.572 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T14:32:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T14:32:57.760 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T14:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T14:32:57.920 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T14:32:58.051 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T14:32:58.112 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T14:32:58.117 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T14:32:58.315 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 17/138 2022-11-27T14:32:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T14:32:58.459 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T14:32:58.522 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T14:32:58.571 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T14:32:58.600 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T14:32:58.718 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T14:32:58.807 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 5/138 2022-11-27T14:32:58.981 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T14:32:59.333 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T14:32:59.373 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T14:32:59.534 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T14:32:59.614 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T14:33:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T14:33:00.132 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T14:33:00.222 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T14:33:00.334 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T14:33:00.382 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T14:33:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T14:33:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T14:33:00.894 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T14:33:01.004 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T14:33:01.141 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T14:33:01.308 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T14:33:01.314 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 22/138 2022-11-27T14:33:01.470 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 23/138 2022-11-27T14:33:01.682 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T14:33:01.750 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/138 2022-11-27T14:33:01.934 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T14:33:02.060 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/138 2022-11-27T14:33:02.130 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T14:33:02.235 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T14:33:02.335 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T14:33:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T14:33:02.457 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T14:33:02.532 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T14:33:02.635 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 17/138 2022-11-27T14:33:02.661 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T14:33:02.738 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T14:33:02.881 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T14:33:02.938 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T14:33:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T14:33:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T14:33:03.821 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T14:33:03.879 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/138 2022-11-27T14:33:04.286 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/138 2022-11-27T14:33:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T14:33:04.577 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/138 2022-11-27T14:33:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T14:33:04.759 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T14:33:05.296 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T14:33:05.601 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 22/138 2022-11-27T14:33:05.757 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 23/138 2022-11-27T14:33:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/138 2022-11-27T14:33:06.349 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/138 2022-11-27T14:33:06.474 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T14:33:06.592 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T14:33:06.802 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T14:33:06.850 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/138 2022-11-27T14:33:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T14:33:07.052 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T14:33:07.188 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T14:33:07.267 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T14:33:07.515 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T14:33:07.752 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T14:33:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T14:33:08.151 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/138 2022-11-27T14:33:08.153 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 40/138 2022-11-27T14:33:08.311 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 41/138 2022-11-27T14:33:08.540 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/138 2022-11-27T14:33:08.547 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/138 2022-11-27T14:33:08.764 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 43/138 2022-11-27T14:33:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/138 2022-11-27T14:33:09.013 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 44/138 2022-11-27T14:33:09.062 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T14:33:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 45/138 2022-11-27T14:33:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 46/138 2022-11-27T14:33:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 47/138 2022-11-27T14:33:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 48/138 2022-11-27T14:33:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 49/138 2022-11-27T14:33:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T14:33:10.803 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 50/138 2022-11-27T14:33:11.032 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 51/138 2022-11-27T14:33:11.129 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/138 2022-11-27T14:33:11.250 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 52/138 2022-11-27T14:33:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T14:33:11.501 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 53/138 2022-11-27T14:33:11.579 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T14:33:11.643 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 54/138 2022-11-27T14:33:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T14:33:11.985 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/138 2022-11-27T14:33:12.089 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T14:33:12.261 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/138 2022-11-27T14:33:12.339 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T14:33:12.465 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 40/138 2022-11-27T14:33:12.542 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/138 2022-11-27T14:33:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 41/138 2022-11-27T14:33:12.819 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/138 2022-11-27T14:33:12.894 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/138 2022-11-27T14:33:13.075 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 59/138 2022-11-27T14:33:13.100 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 43/138 2022-11-27T14:33:13.291 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.7-15.el8.x86_64 60/138 2022-11-27T14:33:13.340 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 44/138 2022-11-27T14:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T14:33:13.569 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T14:33:13.729 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 45/138 2022-11-27T14:33:13.774 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T14:33:13.835 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T14:33:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 46/138 2022-11-27T14:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-3.el8.x86_64 63/138 2022-11-27T14:33:14.206 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 64/138 2022-11-27T14:33:14.236 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 47/138 2022-11-27T14:33:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 48/138 2022-11-27T14:33:14.488 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 65/138 2022-11-27T14:33:14.715 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.1-7.el8.noarch 66/138 2022-11-27T14:33:14.730 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 49/138 2022-11-27T14:33:14.925 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 67/138 2022-11-27T14:33:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 50/138 2022-11-27T14:33:15.140 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T14:33:15.284 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 51/138 2022-11-27T14:33:15.336 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T14:33:15.469 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 52/138 2022-11-27T14:33:15.657 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T14:33:15.713 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 53/138 2022-11-27T14:33:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 54/138 2022-11-27T14:33:15.897 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T14:33:16.087 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T14:33:16.234 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T14:33:16.251 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/138 2022-11-27T14:33:16.433 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T14:33:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/138 2022-11-27T14:33:16.668 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T14:33:16.738 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/138 2022-11-27T14:33:16.867 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T14:33:16.996 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/138 2022-11-27T14:33:17.051 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T14:33:17.195 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T14:33:17.261 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 59/138 2022-11-27T14:33:17.443 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T14:33:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.7-15.el8.x86_64 60/138 2022-11-27T14:33:17.670 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T14:33:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T14:33:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T14:33:17.854 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T14:33:18.002 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T14:33:18.079 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T14:33:18.150 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T14:33:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-3.el8.x86_64 63/138 2022-11-27T14:33:18.455 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T14:33:18.492 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 64/138 2022-11-27T14:33:18.647 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T14:33:18.741 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 65/138 2022-11-27T14:33:18.888 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T14:33:18.985 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.1-7.el8.noarch 66/138 2022-11-27T14:33:19.161 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 67/138 2022-11-27T14:33:19.360 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T14:33:19.530 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T14:33:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T14:33:19.963 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T14:33:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T14:33:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T14:33:20.258 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T14:33:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T14:33:20.524 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T14:33:20.634 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T14:33:20.863 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T14:33:20.871 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T14:33:21.062 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T14:33:21.173 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T14:33:21.246 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T14:33:21.321 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T14:33:21.440 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T14:33:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T14:33:21.870 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T14:33:22.056 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T14:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T14:33:22.280 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T14:33:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T14:33:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T14:33:22.491 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T14:33:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T14:33:22.699 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T14:33:22.906 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T14:33:23.986 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T14:33:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T14:33:24.515 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T14:33:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T14:33:25.150 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T14:33:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T14:33:26.128 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T14:33:26.308 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T14:33:26.385 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T14:33:26.600 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T14:33:26.870 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T14:33:27.141 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T14:33:27.698 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T14:33:27.989 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T14:33:28.073 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T14:33:28.238 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T14:33:30.466 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T14:33:30.708 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T14:33:31.262 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T14:33:31.538 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T14:33:31.605 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T14:33:31.732 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T14:33:47.125 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T14:33:47.126 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2022-11-27T14:33:47.126 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2022-11-27T14:33:47.126 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /dev 2022-11-27T14:33:47.127 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /run 2022-11-27T14:33:47.127 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2022-11-27T14:33:47.127 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2022-11-27T14:33:47.127 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2022-11-27T14:33:47.128 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2022-11-27T14:33:47.128 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /dev 2022-11-27T14:33:47.128 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:33:47.518 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T14:33:48.999 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T14:33:49.219 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T14:33:49.437 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T14:33:49.635 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T14:33:50.015 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T14:33:50.218 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T14:33:50.236 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T14:33:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T14:33:50.421 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2022-11-27T14:33:50.422 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2022-11-27T14:33:50.422 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2022-11-27T14:33:50.422 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /run 2022-11-27T14:33:50.422 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2022-11-27T14:33:50.423 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2022-11-27T14:33:50.423 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2022-11-27T14:33:50.423 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2022-11-27T14:33:50.423 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2022-11-27T14:33:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:33:50.770 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T14:33:51.786 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T14:33:51.963 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T14:33:52.090 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T14:33:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T14:33:52.271 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T14:33:52.450 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T14:33:52.467 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T14:33:52.666 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T14:33:52.675 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T14:33:52.828 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T14:33:52.976 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T14:33:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T14:33:53.419 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T14:33:53.436 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T14:33:53.935 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T14:33:54.083 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T14:33:54.173 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T14:33:54.569 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T14:33:54.808 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T14:33:54.843 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T14:33:55.010 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T14:33:55.034 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T14:33:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T14:33:55.151 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T14:33:55.307 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T14:33:55.480 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T14:33:55.660 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T14:33:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T14:33:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T14:33:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T14:33:57.026 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T14:33:57.060 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T14:33:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T14:33:57.241 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T14:33:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T14:33:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T14:33:57.409 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T14:33:57.414 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T14:33:57.754 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T14:33:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T14:33:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T14:33:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T14:33:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T14:33:58.193 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T14:33:58.204 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T14:33:59.943 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T14:34:00.144 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T14:34:00.177 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T14:34:00.359 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T14:34:00.401 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T14:34:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T14:34:00.938 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T14:34:01.126 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T14:34:02.987 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T14:34:03.300 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T14:34:03.474 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T14:34:03.663 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T14:34:03.963 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/138 2022-11-27T14:34:04.746 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 119/138 2022-11-27T14:34:05.044 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/138 2022-11-27T14:34:05.244 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/138 2022-11-27T14:34:05.419 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T14:34:05.583 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T14:34:06.205 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T14:34:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T14:34:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T14:34:06.868 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T14:34:07.205 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/138 2022-11-27T14:34:07.647 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T14:34:07.932 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T14:34:07.950 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T14:34:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 119/138 2022-11-27T14:34:08.257 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T14:34:08.293 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/138 2022-11-27T14:34:08.439 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T14:34:08.494 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/138 2022-11-27T14:34:08.711 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T14:34:08.752 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T14:34:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T14:34:09.154 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T14:34:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T14:34:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T14:34:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T14:34:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/138 2022-11-27T14:34:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T14:34:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T14:34:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T14:34:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T14:34:10.658 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T14:34:10.659 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T14:34:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T14:34:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T14:34:10.910 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T14:34:11.102 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T14:34:11.103 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T14:34:11.173 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T14:34:11.191 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T14:34:11.370 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T14:34:11.407 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T14:34:11.596 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T14:34:11.656 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T14:34:11.975 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T14:34:12.128 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T14:34:12.128 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T14:34:12.129 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T14:34:12.129 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T14:34:12.129 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T14:34:12.129 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T14:34:12.130 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T14:34:12.130 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T14:34:12.130 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T14:34:12.130 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T14:34:12.131 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T14:34:12.131 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T14:34:12.131 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T14:34:12.131 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T14:34:12.132 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T14:34:12.132 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T14:34:12.132 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T14:34:12.132 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T14:34:12.133 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T14:34:12.133 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T14:34:12.133 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T14:34:12.134 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T14:34:12.134 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T14:34:12.134 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T14:34:12.134 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T14:34:12.135 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T14:34:12.135 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T14:34:12.135 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T14:34:12.135 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T14:34:12.149 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T14:34:12.149 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T14:34:12.150 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T14:34:12.150 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T14:34:12.150 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T14:34:12.150 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T14:34:12.151 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T14:34:12.151 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T14:34:12.151 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T14:34:12.151 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T14:34:12.152 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T14:34:12.152 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T14:34:12.152 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T14:34:12.152 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T14:34:12.153 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T14:34:12.153 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T14:34:12.153 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T14:34:12.153 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T14:34:12.153 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T14:34:12.154 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T14:34:12.154 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T14:34:12.154 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T14:34:12.154 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T14:34:12.154 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T14:34:12.155 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T14:34:12.155 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T14:34:12.155 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T14:34:12.155 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T14:34:12.156 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T14:34:12.156 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T14:34:12.156 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T14:34:12.156 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T14:34:12.157 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T14:34:12.157 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T14:34:12.158 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T14:34:12.158 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T14:34:12.159 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T14:34:12.159 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T14:34:12.160 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T14:34:12.160 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T14:34:12.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T14:34:12.161 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T14:34:12.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T14:34:12.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 72/138 2022-11-27T14:34:12.162 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.0.4-10.el8.x86_64 73/138 2022-11-27T14:34:12.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 74/138 2022-11-27T14:34:12.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 75/138 2022-11-27T14:34:12.163 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 76/138 2022-11-27T14:34:12.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/138 2022-11-27T14:34:12.164 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/138 2022-11-27T14:34:12.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 79/138 2022-11-27T14:34:12.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 80/138 2022-11-27T14:34:12.165 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 81/138 2022-11-27T14:34:12.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 82/138 2022-11-27T14:34:12.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 83/138 2022-11-27T14:34:12.166 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 84/138 2022-11-27T14:34:12.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 85/138 2022-11-27T14:34:12.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/138 2022-11-27T14:34:12.167 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/138 2022-11-27T14:34:12.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/138 2022-11-27T14:34:12.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.1-9.el8.x86_64 89/138 2022-11-27T14:34:12.168 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 90/138 2022-11-27T14:34:12.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-3.el8.x86_64 91/138 2022-11-27T14:34:12.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 92/138 2022-11-27T14:34:12.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.1-7.el8.noarch 93/138 2022-11-27T14:34:12.169 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 94/138 2022-11-27T14:34:12.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 95/138 2022-11-27T14:34:12.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 96/138 2022-11-27T14:34:12.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/138 2022-11-27T14:34:12.170 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 98/138 2022-11-27T14:34:12.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 99/138 2022-11-27T14:34:12.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 100/138 2022-11-27T14:34:12.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 101/138 2022-11-27T14:34:12.171 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/138 2022-11-27T14:34:12.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 103/138 2022-11-27T14:34:12.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 104/138 2022-11-27T14:34:12.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 105/138 2022-11-27T14:34:12.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/138 2022-11-27T14:34:12.172 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 107/138 2022-11-27T14:34:12.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 108/138 2022-11-27T14:34:12.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 109/138 2022-11-27T14:34:12.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 110/138 2022-11-27T14:34:12.173 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 111/138 2022-11-27T14:34:12.174 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 112/138 2022-11-27T14:34:12.174 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/138 2022-11-27T14:34:12.174 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 114/138 2022-11-27T14:34:12.174 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 115/138 2022-11-27T14:34:12.174 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/138 2022-11-27T14:34:12.175 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 117/138 2022-11-27T14:34:12.175 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/138 2022-11-27T14:34:12.175 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 119/138 2022-11-27T14:34:12.175 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 120/138 2022-11-27T14:34:12.176 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 121/138 2022-11-27T14:34:12.176 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 122/138 2022-11-27T14:34:12.176 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 123/138 2022-11-27T14:34:12.176 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 124/138 2022-11-27T14:34:12.177 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 125/138 2022-11-27T14:34:12.177 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 126/138 2022-11-27T14:34:12.177 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/138 2022-11-27T14:34:12.177 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/138 2022-11-27T14:34:12.178 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T14:34:12.178 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T14:34:12.178 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T14:34:12.179 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T14:34:12.179 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T14:34:12.179 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T14:34:12.180 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T14:34:12.180 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T14:34:12.180 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T14:34:12.367 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T14:34:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T14:34:12.889 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T14:34:12.898 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T14:34:13.083 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/138 2022-11-27T14:34:13.306 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T14:34:13.382 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T14:34:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T14:34:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T14:34:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T14:34:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T14:34:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T14:34:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T14:34:14.295 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T14:34:14.295 INFO:teuthology.orchestra.run.smithi137.stdout:Installed products updated. 2022-11-27T14:34:14.317 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T14:34:14.318 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T14:34:14.561 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:14.562 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2022-11-27T14:34:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T14:34:14.562 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T14:34:14.563 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T14:34:14.564 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2022-11-27T14:34:14.564 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T14:34:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T14:34:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T14:34:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.565 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.566 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.567 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.568 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.569 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.570 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.570 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.571 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.571 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.572 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.572 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.573 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:14.573 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T14:34:14.573 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T14:34:14.574 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T14:34:14.574 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T14:34:14.574 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-3.el8.x86_64 2022-11-27T14:34:14.574 INFO:teuthology.orchestra.run.smithi137.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T14:34:14.575 INFO:teuthology.orchestra.run.smithi137.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T14:34:14.575 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T14:34:14.575 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.575 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.576 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.576 INFO:teuthology.orchestra.run.smithi137.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T14:34:14.576 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T14:34:14.576 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T14:34:14.577 INFO:teuthology.orchestra.run.smithi137.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T14:34:14.577 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T14:34:14.577 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T14:34:14.577 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.578 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.578 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T14:34:14.578 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.578 INFO:teuthology.orchestra.run.smithi137.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T14:34:14.579 INFO:teuthology.orchestra.run.smithi137.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T14:34:14.579 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T14:34:14.579 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T14:34:14.580 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T14:34:14.580 INFO:teuthology.orchestra.run.smithi137.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T14:34:14.581 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T14:34:14.581 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T14:34:14.581 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T14:34:14.582 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T14:34:14.582 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T14:34:14.582 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T14:34:14.582 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T14:34:14.583 INFO:teuthology.orchestra.run.smithi137.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T14:34:14.583 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T14:34:14.583 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T14:34:14.583 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T14:34:14.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T14:34:14.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T14:34:14.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T14:34:14.584 INFO:teuthology.orchestra.run.smithi137.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T14:34:14.585 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T14:34:14.585 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.585 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.585 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.585 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T14:34:14.586 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T14:34:14.586 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T14:34:14.586 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T14:34:14.586 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T14:34:14.587 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T14:34:14.587 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T14:34:14.587 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T14:34:14.587 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T14:34:14.588 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T14:34:14.588 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T14:34:14.588 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T14:34:14.588 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T14:34:14.589 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T14:34:14.589 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T14:34:14.589 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T14:34:14.589 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T14:34:14.589 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T14:34:14.590 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T14:34:14.590 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T14:34:14.590 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T14:34:14.591 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T14:34:14.591 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T14:34:14.591 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T14:34:14.592 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T14:34:14.592 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T14:34:14.592 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T14:34:14.593 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T14:34:14.593 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T14:34:14.593 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.593 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.594 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T14:34:14.594 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T14:34:14.594 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.594 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T14:34:14.595 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T14:34:14.595 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T14:34:14.595 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T14:34:14.595 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T14:34:14.596 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T14:34:14.596 INFO:teuthology.orchestra.run.smithi137.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T14:34:14.596 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T14:34:14.596 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T14:34:14.597 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T14:34:14.597 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T14:34:14.597 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T14:34:14.597 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T14:34:14.598 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.598 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.598 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:14.598 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T14:34:14.599 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T14:34:14.599 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T14:34:14.599 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T14:34:14.599 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T14:34:14.601 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T14:34:14.601 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T14:34:14.601 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:14.602 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2022-11-27T14:34:14.748 DEBUG:teuthology.parallel:result is None 2022-11-27T14:34:14.796 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T14:34:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T14:34:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T14:34:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T14:34:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T14:34:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T14:34:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T14:34:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T14:34:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T14:34:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T14:34:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T14:34:15.367 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T14:34:15.367 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T14:34:15.367 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T14:34:15.367 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T14:34:15.367 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T14:34:15.368 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T14:34:15.368 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T14:34:15.368 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T14:34:15.368 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T14:34:15.369 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T14:34:15.369 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T14:34:15.369 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T14:34:15.369 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T14:34:15.369 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T14:34:15.370 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T14:34:15.370 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T14:34:15.370 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T14:34:15.371 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T14:34:15.371 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T14:34:15.371 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T14:34:15.371 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T14:34:15.371 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T14:34:15.372 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T14:34:15.372 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T14:34:15.372 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T14:34:15.372 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T14:34:15.373 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T14:34:15.373 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T14:34:15.373 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T14:34:15.373 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T14:34:15.373 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T14:34:15.374 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T14:34:15.374 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T14:34:15.374 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T14:34:15.374 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T14:34:15.374 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T14:34:15.375 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T14:34:15.375 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T14:34:15.375 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T14:34:15.375 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T14:34:15.376 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T14:34:15.376 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T14:34:15.376 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T14:34:15.376 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T14:34:15.376 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T14:34:15.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T14:34:15.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T14:34:15.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T14:34:15.377 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T14:34:15.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T14:34:15.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T14:34:15.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T14:34:15.378 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T14:34:15.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T14:34:15.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T14:34:15.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T14:34:15.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T14:34:15.379 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T14:34:15.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T14:34:15.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T14:34:15.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T14:34:15.380 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T14:34:15.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 72/138 2022-11-27T14:34:15.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.0.4-10.el8.x86_64 73/138 2022-11-27T14:34:15.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 74/138 2022-11-27T14:34:15.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 75/138 2022-11-27T14:34:15.381 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 76/138 2022-11-27T14:34:15.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/138 2022-11-27T14:34:15.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/138 2022-11-27T14:34:15.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 79/138 2022-11-27T14:34:15.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 80/138 2022-11-27T14:34:15.382 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 81/138 2022-11-27T14:34:15.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 82/138 2022-11-27T14:34:15.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 83/138 2022-11-27T14:34:15.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 84/138 2022-11-27T14:34:15.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 85/138 2022-11-27T14:34:15.383 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/138 2022-11-27T14:34:15.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/138 2022-11-27T14:34:15.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/138 2022-11-27T14:34:15.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.1-9.el8.x86_64 89/138 2022-11-27T14:34:15.384 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 90/138 2022-11-27T14:34:15.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-3.el8.x86_64 91/138 2022-11-27T14:34:15.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 92/138 2022-11-27T14:34:15.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.1-7.el8.noarch 93/138 2022-11-27T14:34:15.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 94/138 2022-11-27T14:34:15.385 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 95/138 2022-11-27T14:34:15.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 96/138 2022-11-27T14:34:15.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/138 2022-11-27T14:34:15.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 98/138 2022-11-27T14:34:15.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 99/138 2022-11-27T14:34:15.386 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 100/138 2022-11-27T14:34:15.387 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 101/138 2022-11-27T14:34:15.387 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/138 2022-11-27T14:34:15.387 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 103/138 2022-11-27T14:34:15.387 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 104/138 2022-11-27T14:34:15.388 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 105/138 2022-11-27T14:34:15.388 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/138 2022-11-27T14:34:15.388 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 107/138 2022-11-27T14:34:15.388 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 108/138 2022-11-27T14:34:15.388 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 109/138 2022-11-27T14:34:15.389 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 110/138 2022-11-27T14:34:15.389 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 111/138 2022-11-27T14:34:15.389 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 112/138 2022-11-27T14:34:15.389 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/138 2022-11-27T14:34:15.389 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 114/138 2022-11-27T14:34:15.390 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 115/138 2022-11-27T14:34:15.390 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/138 2022-11-27T14:34:15.390 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 117/138 2022-11-27T14:34:15.390 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/138 2022-11-27T14:34:15.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 119/138 2022-11-27T14:34:15.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 120/138 2022-11-27T14:34:15.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 121/138 2022-11-27T14:34:15.391 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 122/138 2022-11-27T14:34:15.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 123/138 2022-11-27T14:34:15.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 124/138 2022-11-27T14:34:15.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 125/138 2022-11-27T14:34:15.392 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 126/138 2022-11-27T14:34:15.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/138 2022-11-27T14:34:15.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/138 2022-11-27T14:34:15.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T14:34:15.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T14:34:15.393 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T14:34:15.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T14:34:15.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T14:34:15.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T14:34:15.394 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T14:34:15.395 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T14:34:15.395 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T14:34:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T14:34:17.453 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2022-11-27T14:34:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:17.710 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2022-11-27T14:34:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T14:34:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T14:34:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T14:34:17.711 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2022-11-27T14:34:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T14:34:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T14:34:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T14:34:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T14:34:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T14:34:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T14:34:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T14:34:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T14:34:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-3.el8.x86_64 2022-11-27T14:34:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T14:34:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T14:34:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T14:34:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T14:34:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T14:34:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T14:34:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T14:34:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T14:34:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T14:34:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T14:34:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T14:34:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T14:34:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T14:34:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T14:34:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T14:34:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T14:34:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T14:34:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T14:34:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T14:34:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T14:34:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T14:34:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T14:34:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T14:34:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T14:34:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T14:34:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T14:34:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T14:34:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T14:34:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T14:34:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T14:34:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T14:34:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T14:34:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T14:34:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T14:34:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T14:34:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T14:34:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T14:34:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T14:34:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T14:34:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T14:34:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T14:34:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T14:34:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T14:34:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T14:34:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T14:34:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T14:34:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T14:34:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T14:34:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T14:34:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T14:34:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T14:34:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T14:34:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T14:34:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T14:34:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T14:34:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T14:34:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T14:34:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T14:34:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T14:34:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T14:34:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T14:34:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T14:34:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T14:34:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T14:34:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T14:34:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T14:34:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T14:34:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T14:34:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T14:34:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T14:34:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T14:34:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T14:34:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T14:34:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T14:34:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T14:34:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T14:34:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T14:34:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T14:34:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T14:34:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T14:34:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T14:34:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T14:34:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T14:34:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:17.742 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2022-11-27T14:34:17.914 DEBUG:teuthology.parallel:result is None 2022-11-27T14:34:17.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo lsb_release -is 2022-11-27T14:34:17.971 INFO:teuthology.orchestra.run.smithi099.stdout:RedHatEnterprise 2022-11-27T14:34:17.972 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T14:34:17.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:34:18.127 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T14:34:18.149 INFO:teuthology.orchestra.run.smithi099.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T14:34:18.150 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T14:34:18.151 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T14:34:18.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo lsb_release -is 2022-11-27T14:34:18.216 INFO:teuthology.orchestra.run.smithi137.stdout:RedHatEnterprise 2022-11-27T14:34:18.218 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T14:34:18.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T14:34:18.375 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T14:34:18.399 INFO:teuthology.orchestra.run.smithi137.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T14:34:18.399 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T14:34:18.400 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T14:34:18.401 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-27T14:34:18.401 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:18.402 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T14:34:18.429 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:18.429 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T14:34:18.469 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-27T14:34:18.470 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:18.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T14:34:18.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T14:34:18.563 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:18.564 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T14:34:18.595 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T14:34:18.665 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-27T14:34:18.666 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:18.666 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T14:34:18.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T14:34:18.759 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:18.760 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T14:34:18.789 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T14:34:18.858 INFO:teuthology.run_tasks:Running task ceph... 2022-11-27T14:34:18.939 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-27T14:34:18.940 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2022-11-27T14:34:18.943 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /var/log/ceph 2022-11-27T14:34:18.972 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-27T14:34:18.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T14:34:19.010 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T14:34:19.040 INFO:tasks.ceph:Creating extra log directories... 2022-11-27T14:34:19.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T14:34:19.077 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T14:34:19.120 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-11-27T14:34:19.121 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-11-27T14:34:19.122 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-11-27T14:34:19.123 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:19.123 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T14:34:19.159 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T14:34:19.272 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T14:34:19.299 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:19.299 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T14:34:19.317 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T14:34:19.430 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T14:34:19.463 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-27T14:34:19.463 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-27T14:34:19.464 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088565', 'branch': 'main', 'description': 'fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7088565', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAp1TozT0uutM8HOCPzxEUSXZmhnkJgIflw9VEshFT4NgBcl8qb5zeruryssm+11PL0ZzY0QFAJKRCeRNotEZM=', 'smithi137.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0i848JIwZQdAX9/JFm15VYV34au3I21waYzUFCIZXYbgTm1YraDScyXY6UBdStOjwsIGb0D5/dgJK1MOwpAdcyXzZC2VzgxxhCTVxjrttdvpTM0LXq5ZEUxcdPgInjAH93llCZdUvVRr6a7KA2KUDHEN9CjQ5TgrgIoV+GsJN5w7JM8eJfMUBEaabd2Plp36YccCUbtxf8BgRYkcZCU+D8zODIEonvdQoIelNsTOwOP8qZq3xLLHCbniHc3ulWZGq6RqfyMGxkoJeqHI+e4X9YMe0qHz2Vl/Lubu4xwlUyFzoge19FREG2Uas5syv+0AmQz1c0OK44W57Iy28IARekPOdPm0TaaaA6cJEcCh1+D7wTqZG1kIEZBdqfoxoMXkDUSAqsvY/meDVnTgAQvE9yiadF1QIjz/1k9RhiYhZA3CeRfOQsYtYTBpz+DB0uz9ZvjOkegY2bSZbB39LagOk+73kB4KSijezWCjOn3S8I/QBt9lnJ75PqZIsknJWulc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_multifs_auth']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-23_17:24:26', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.11998'} 2022-11-27T14:34:19.465 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T14:34:19.468 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T14:34:19.515 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T14:34:19.525 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T14:34:19.581 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:19.581 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T14:34:19.606 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T14:34:19.606 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2022-11-27T14:34:19.663 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T14:34:19.663 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T14:34:19.663 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 358317 Links: 1 2022-11-27T14:34:19.664 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T14:34:19.664 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T14:34:19.664 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2022-11-27 14:34:15.053845508 +0000 2022-11-27T14:34:19.664 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2022-11-27 14:31:11.056399740 +0000 2022-11-27T14:34:19.664 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2022-11-27 14:31:11.056399740 +0000 2022-11-27T14:34:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2022-11-27T14:34:19.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T14:34:19.728 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2022-11-27T14:34:19.728 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2022-11-27T14:34:19.729 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.00018486 s, 2.8 MB/s 2022-11-27T14:34:19.730 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T14:34:19.788 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2022-11-27T14:34:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T14:34:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T14:34:19.846 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 354233 Links: 1 2022-11-27T14:34:19.846 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T14:34:19.846 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T14:34:19.846 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2022-11-27 14:34:15.053845508 +0000 2022-11-27T14:34:19.847 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2022-11-27 14:31:11.603386837 +0000 2022-11-27T14:34:19.847 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2022-11-27 14:31:11.603386837 +0000 2022-11-27T14:34:19.847 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2022-11-27T14:34:19.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T14:34:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2022-11-27T14:34:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2022-11-27T14:34:19.911 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000199636 s, 2.6 MB/s 2022-11-27T14:34:19.913 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T14:34:19.971 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2022-11-27T14:34:20.028 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T14:34:20.028 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T14:34:20.028 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 369681 Links: 1 2022-11-27T14:34:20.029 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T14:34:20.029 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T14:34:20.029 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2022-11-27 14:34:15.053845508 +0000 2022-11-27T14:34:20.030 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2022-11-27 14:31:12.166373557 +0000 2022-11-27T14:34:20.030 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2022-11-27 14:31:12.166373557 +0000 2022-11-27T14:34:20.030 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2022-11-27T14:34:20.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T14:34:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2022-11-27T14:34:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2022-11-27T14:34:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000192856 s, 2.7 MB/s 2022-11-27T14:34:20.096 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T14:34:20.153 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2022-11-27T14:34:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T14:34:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T14:34:20.211 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 357016 Links: 1 2022-11-27T14:34:20.211 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T14:34:20.211 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T14:34:20.212 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2022-11-27 14:34:15.053845508 +0000 2022-11-27T14:34:20.212 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2022-11-27 14:31:12.733360182 +0000 2022-11-27T14:34:20.212 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2022-11-27 14:31:12.733360182 +0000 2022-11-27T14:34:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2022-11-27T14:34:20.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T14:34:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2022-11-27T14:34:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2022-11-27T14:34:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000216985 s, 2.4 MB/s 2022-11-27T14:34:20.277 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T14:34:20.335 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:20.335 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:20.336 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T14:34:20.355 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T14:34:20.355 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2022-11-27T14:34:20.414 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T14:34:20.414 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T14:34:20.414 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 364808 Links: 1 2022-11-27T14:34:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T14:34:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T14:34:20.415 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2022-11-27 14:34:11.817818103 +0000 2022-11-27T14:34:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2022-11-27 14:31:11.063110498 +0000 2022-11-27T14:34:20.416 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2022-11-27 14:31:11.063110498 +0000 2022-11-27T14:34:20.416 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2022-11-27T14:34:20.416 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T14:34:20.481 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2022-11-27T14:34:20.482 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2022-11-27T14:34:20.482 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000239419 s, 2.1 MB/s 2022-11-27T14:34:20.483 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T14:34:20.542 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2022-11-27T14:34:20.600 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T14:34:20.600 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T14:34:20.601 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 364974 Links: 1 2022-11-27T14:34:20.601 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T14:34:20.601 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T14:34:20.601 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2022-11-27 14:34:11.817818103 +0000 2022-11-27T14:34:20.602 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2022-11-27 14:31:11.612100973 +0000 2022-11-27T14:34:20.602 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2022-11-27 14:31:11.612100973 +0000 2022-11-27T14:34:20.602 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2022-11-27T14:34:20.603 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T14:34:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2022-11-27T14:34:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2022-11-27T14:34:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000204488 s, 2.5 MB/s 2022-11-27T14:34:20.668 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T14:34:20.727 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2022-11-27T14:34:20.784 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T14:34:20.785 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T14:34:20.785 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 351162 Links: 1 2022-11-27T14:34:20.785 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T14:34:20.786 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T14:34:20.786 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2022-11-27 14:34:11.817818103 +0000 2022-11-27T14:34:20.786 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2022-11-27 14:31:12.194090875 +0000 2022-11-27T14:34:20.786 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2022-11-27 14:31:12.194090875 +0000 2022-11-27T14:34:20.787 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2022-11-27T14:34:20.787 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T14:34:20.851 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2022-11-27T14:34:20.851 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2022-11-27T14:34:20.851 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000194785 s, 2.6 MB/s 2022-11-27T14:34:20.853 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T14:34:20.912 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2022-11-27T14:34:20.969 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T14:34:20.970 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T14:34:20.970 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 6h/6d Inode: 365775 Links: 1 2022-11-27T14:34:20.970 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T14:34:20.970 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T14:34:20.971 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2022-11-27 14:34:11.817818103 +0000 2022-11-27T14:34:20.971 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2022-11-27 14:31:12.742081367 +0000 2022-11-27T14:34:20.971 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2022-11-27 14:31:12.742081367 +0000 2022-11-27T14:34:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2022-11-27T14:34:20.972 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T14:34:21.034 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2022-11-27T14:34:21.034 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2022-11-27T14:34:21.034 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.00022249 s, 2.3 MB/s 2022-11-27T14:34:21.035 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T14:34:21.094 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:21.094 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T14:34:21.094 INFO:tasks.ceph:Generating config... 2022-11-27T14:34:21.096 INFO:tasks.ceph:[client] client mount timeout = 600 2022-11-27T14:34:21.096 INFO:tasks.ceph:[client] debug client = 20 2022-11-27T14:34:21.097 INFO:tasks.ceph:[client] debug ms = 1 2022-11-27T14:34:21.097 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-11-27T14:34:21.097 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-11-27T14:34:21.098 INFO:tasks.ceph:[global] ms die on skipped message = False 2022-11-27T14:34:21.098 INFO:tasks.ceph:[mds] debug mds = 20 2022-11-27T14:34:21.098 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-11-27T14:34:21.099 INFO:tasks.ceph:[mds] debug ms = 1 2022-11-27T14:34:21.099 INFO:tasks.ceph:[mds] mds debug frag = True 2022-11-27T14:34:21.099 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-11-27T14:34:21.099 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-11-27T14:34:21.100 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-11-27T14:34:21.100 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-11-27T14:34:21.100 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-11-27T14:34:21.101 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-11-27T14:34:21.101 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-27T14:34:21.101 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-27T14:34:21.102 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-27T14:34:21.102 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-27T14:34:21.102 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-27T14:34:21.102 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-11-27T14:34:21.103 INFO:tasks.ceph:[osd] bdev async discard = True 2022-11-27T14:34:21.103 INFO:tasks.ceph:[osd] bdev enable discard = True 2022-11-27T14:34:21.103 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2022-11-27T14:34:21.104 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-27T14:34:21.104 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-27T14:34:21.104 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2022-11-27T14:34:21.104 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2022-11-27T14:34:21.105 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-27T14:34:21.105 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-27T14:34:21.105 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2022-11-27T14:34:21.106 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-27T14:34:21.106 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-27T14:34:21.106 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-27T14:34:21.107 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-11-27T14:34:21.107 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-11-27T14:34:21.107 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-11-27T14:34:21.108 INFO:tasks.ceph:Setting up mon.a... 2022-11-27T14:34:21.108 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-27T14:34:21.170 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.keyring 2022-11-27T14:34:21.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-27T14:34:21.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T14:34:21.249 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.99'), ('mon.b', '172.21.15.137'), ('mon.c', '[v2:172.21.15.137:3301,v1:172.21.15.137:6790]')] 2022-11-27T14:34:21.250 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.99,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-11-27T14:34:21.251 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:21.251 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T14:34:21.306 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.99 --add b 172.21.15.137 --addv c '[v2:172.21.15.137:3301,v1:172.21.15.137:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:21.394 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:21.394 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: generated fsid 1f55be23-348c-4a1c-92de-37e343596508 2022-11-27T14:34:21.394 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = pacific 2022-11-27T14:34:21.395 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 0 2022-11-27T14:34:21.395 INFO:teuthology.orchestra.run.smithi099.stdout:fsid 1f55be23-348c-4a1c-92de-37e343596508 2022-11-27T14:34:21.395 INFO:teuthology.orchestra.run.smithi099.stdout:last_changed 2022-11-27T14:34:21.393823+0000 2022-11-27T14:34:21.396 INFO:teuthology.orchestra.run.smithi099.stdout:created 2022-11-27T14:34:21.393823+0000 2022-11-27T14:34:21.396 INFO:teuthology.orchestra.run.smithi099.stdout:min_mon_release 16 (pacific) 2022-11-27T14:34:21.397 INFO:teuthology.orchestra.run.smithi099.stdout:election_strategy: 1 2022-11-27T14:34:21.397 INFO:teuthology.orchestra.run.smithi099.stdout:0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2022-11-27T14:34:21.397 INFO:teuthology.orchestra.run.smithi099.stdout:1: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2022-11-27T14:34:21.398 INFO:teuthology.orchestra.run.smithi099.stdout:2: [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] mon.c 2022-11-27T14:34:21.398 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-27T14:34:21.401 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T14:34:21.418 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1f55be23-348c-4a1c-92de-37e343596508... 2022-11-27T14:34:21.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T14:34:21.460 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T14:34:21.503 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2022-11-27T14:34:21.503 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2022-11-27T14:34:21.503 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T14:34:21.504 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2022-11-27T14:34:21.504 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.504 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2022-11-27T14:34:21.504 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.504 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2022-11-27T14:34:21.505 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.505 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2022-11-27T14:34:21.505 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2022-11-27T14:34:21.505 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.506 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2022-11-27T14:34:21.506 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2022-11-27T14:34:21.506 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.506 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T14:34:21.506 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2022-11-27T14:34:21.507 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.507 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T14:34:21.507 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T14:34:21.507 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.508 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2022-11-27T14:34:21.508 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.508 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2022-11-27T14:34:21.508 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2022-11-27T14:34:21.509 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2022-11-27T14:34:21.509 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2022-11-27T14:34:21.509 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2022-11-27T14:34:21.509 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2022-11-27T14:34:21.510 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2022-11-27T14:34:21.510 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T14:34:21.510 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T14:34:21.511 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2022-11-27T14:34:21.511 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.511 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T14:34:21.511 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.511 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2022-11-27T14:34:21.512 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.512 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2022-11-27T14:34:21.512 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.512 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2022-11-27T14:34:21.513 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2022-11-27T14:34:21.513 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2022-11-27T14:34:21.513 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.99,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]" 2022-11-27T14:34:21.513 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on skipped message = False 2022-11-27T14:34:21.513 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 1f55be23-348c-4a1c-92de-37e343596508 2022-11-27T14:34:21.514 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.514 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2022-11-27T14:34:21.514 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2022-11-27T14:34:21.514 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.514 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2022-11-27T14:34:21.515 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2022-11-27T14:34:21.515 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.515 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2022-11-27T14:34:21.515 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2022-11-27T14:34:21.516 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.516 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2022-11-27T14:34:21.516 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2022-11-27T14:34:21.516 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2022-11-27T14:34:21.516 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.517 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2022-11-27T14:34:21.517 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2022-11-27T14:34:21.517 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.517 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2022-11-27T14:34:21.517 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.518 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2022-11-27T14:34:21.518 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.518 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2022-11-27T14:34:21.518 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.518 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2022-11-27T14:34:21.519 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2022-11-27T14:34:21.519 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.519 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2022-11-27T14:34:21.519 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2022-11-27T14:34:21.520 INFO:teuthology.orchestra.run.smithi099.stdout: bdev async discard = True 2022-11-27T14:34:21.520 INFO:teuthology.orchestra.run.smithi099.stdout: bdev enable discard = True 2022-11-27T14:34:21.520 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore allocator = bitmap 2022-11-27T14:34:21.520 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2022-11-27T14:34:21.520 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2022-11-27T14:34:21.521 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2022-11-27T14:34:21.521 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2022-11-27T14:34:21.521 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-11-27T14:34:21.521 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2022-11-27T14:34:21.521 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2022-11-27T14:34:21.522 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T14:34:21.522 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2022-11-27T14:34:21.522 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2022-11-27T14:34:21.522 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2022-11-27T14:34:21.523 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2022-11-27T14:34:21.523 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2022-11-27T14:34:21.523 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.523 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2022-11-27T14:34:21.523 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-11-27T14:34:21.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2022-11-27T14:34:21.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2022-11-27T14:34:21.524 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2022-11-27T14:34:21.524 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2022-11-27T14:34:21.524 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2022-11-27T14:34:21.525 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2022-11-27T14:34:21.525 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.525 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2022-11-27T14:34:21.525 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-11-27T14:34:21.525 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2022-11-27T14:34:21.526 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2022-11-27T14:34:21.526 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2022-11-27T14:34:21.526 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2022-11-27T14:34:21.526 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2022-11-27T14:34:21.527 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2022-11-27T14:34:21.527 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2022-11-27T14:34:21.527 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2022-11-27T14:34:21.527 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2022-11-27T14:34:21.527 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.528 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T14:34:21.528 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T14:34:21.528 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T14:34:21.528 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.528 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T14:34:21.529 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T14:34:21.529 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T14:34:21.529 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.529 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2022-11-27T14:34:21.530 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2022-11-27T14:34:21.530 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.530 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2022-11-27T14:34:21.530 INFO:teuthology.orchestra.run.smithi099.stdout: mon op complaint time = 120 2022-11-27T14:34:21.530 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:21.531 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2022-11-27T14:34:21.531 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2022-11-27T14:34:21.531 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2022-11-27T14:34:21.531 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2022-11-27T14:34:21.531 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T14:34:21.532 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T14:34:21.532 INFO:teuthology.orchestra.run.smithi099.stdout: client mount timeout = 600 2022-11-27T14:34:21.532 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2022-11-27T14:34:21.532 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-11-27T14:34:21.532 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2022-11-27T14:34:21.533 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2022-11-27T14:34:21.533 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2022-11-27T14:34:21.533 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2022-11-27T14:34:21.533 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2022-11-27T14:34:21.534 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2022-11-27T14:34:21.534 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2022-11-27T14:34:21.534 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2022-11-27T14:34:21.534 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2022-11-27T14:34:21.534 INFO:teuthology.orchestra.run.smithi099.stdout:[mds] 2022-11-27T14:34:21.535 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds = 20 2022-11-27T14:34:21.535 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds balancer = 20 2022-11-27T14:34:21.535 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2022-11-27T14:34:21.535 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug frag = True 2022-11-27T14:34:21.535 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug scatterstat = True 2022-11-27T14:34:21.536 INFO:teuthology.orchestra.run.smithi099.stdout: mds op complaint time = 180 2022-11-27T14:34:21.536 INFO:teuthology.orchestra.run.smithi099.stdout: mds verify scatter = True 2022-11-27T14:34:21.536 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2022-11-27T14:34:21.536 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2022-11-27T14:34:21.536 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2022-11-27T14:34:21.538 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2022-11-27T14:34:21.539 INFO:teuthology.orchestra.run.smithi137.stdout: chdir = "" 2022-11-27T14:34:21.539 INFO:teuthology.orchestra.run.smithi137.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T14:34:21.539 INFO:teuthology.orchestra.run.smithi137.stdout: auth supported = cephx 2022-11-27T14:34:21.539 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.540 INFO:teuthology.orchestra.run.smithi137.stdout: filestore xattr use omap = true 2022-11-27T14:34:21.540 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.540 INFO:teuthology.orchestra.run.smithi137.stdout: mon clock drift allowed = 1.000 2022-11-27T14:34:21.540 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.540 INFO:teuthology.orchestra.run.smithi137.stdout: osd crush chooseleaf type = 0 2022-11-27T14:34:21.541 INFO:teuthology.orchestra.run.smithi137.stdout: auth debug = true 2022-11-27T14:34:21.541 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.541 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on old message = true 2022-11-27T14:34:21.541 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on bug = true 2022-11-27T14:34:21.541 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.542 INFO:teuthology.orchestra.run.smithi137.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T14:34:21.542 INFO:teuthology.orchestra.run.smithi137.stdout: mon pg warn max object skew = 0 2022-11-27T14:34:21.542 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.542 INFO:teuthology.orchestra.run.smithi137.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T14:34:21.542 INFO:teuthology.orchestra.run.smithi137.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T14:34:21.543 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.543 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default size = 2 2022-11-27T14:34:21.543 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.543 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow primary affinity = true 2022-11-27T14:34:21.544 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow pg remap = true 2022-11-27T14:34:21.544 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on legacy crush tunables = false 2022-11-27T14:34:21.544 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on crush straw calc version zero = false 2022-11-27T14:34:21.544 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on no sortbitwise = false 2022-11-27T14:34:21.544 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on osd down out interval zero = false 2022-11-27T14:34:21.545 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on too few osds = false 2022-11-27T14:34:21.545 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T14:34:21.545 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T14:34:21.545 INFO:teuthology.orchestra.run.smithi137.stdout: mon_allow_pool_size_one = true 2022-11-27T14:34:21.545 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.546 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T14:34:21.546 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.546 INFO:teuthology.orchestra.run.smithi137.stdout: osd default data pool replay window = 5 2022-11-27T14:34:21.546 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.547 INFO:teuthology.orchestra.run.smithi137.stdout: mon allow pool delete = true 2022-11-27T14:34:21.547 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.547 INFO:teuthology.orchestra.run.smithi137.stdout: mon cluster log file level = debug 2022-11-27T14:34:21.547 INFO:teuthology.orchestra.run.smithi137.stdout: debug asserts on shutdown = true 2022-11-27T14:34:21.547 INFO:teuthology.orchestra.run.smithi137.stdout: mon health detail to clog = false 2022-11-27T14:34:21.548 INFO:teuthology.orchestra.run.smithi137.stdout: mon host = "172.21.15.99,172.21.15.137,[v2:172.21.15.137:3301,v1:172.21.15.137:6790]" 2022-11-27T14:34:21.548 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on skipped message = False 2022-11-27T14:34:21.548 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = 1f55be23-348c-4a1c-92de-37e343596508 2022-11-27T14:34:21.548 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.548 INFO:teuthology.orchestra.run.smithi137.stdout:[osd] 2022-11-27T14:34:21.549 INFO:teuthology.orchestra.run.smithi137.stdout: osd journal size = 100 2022-11-27T14:34:21.549 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.549 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub load threshold = 5.0 2022-11-27T14:34:21.549 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub max interval = 600 2022-11-27T14:34:21.550 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.550 INFO:teuthology.orchestra.run.smithi137.stdout: osd recover clone overlap = true 2022-11-27T14:34:21.550 INFO:teuthology.orchestra.run.smithi137.stdout: osd recovery max chunk = 1048576 2022-11-27T14:34:21.550 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.550 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug shutdown = true 2022-11-27T14:34:21.551 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug op order = true 2022-11-27T14:34:21.551 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug verify stray on activate = true 2022-11-27T14:34:21.551 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.551 INFO:teuthology.orchestra.run.smithi137.stdout: osd open classes on start = true 2022-11-27T14:34:21.551 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug pg log writeout = true 2022-11-27T14:34:21.552 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.552 INFO:teuthology.orchestra.run.smithi137.stdout: osd deep scrub update digest min age = 30 2022-11-27T14:34:21.552 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.552 INFO:teuthology.orchestra.run.smithi137.stdout: osd map max advance = 10 2022-11-27T14:34:21.553 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.553 INFO:teuthology.orchestra.run.smithi137.stdout: journal zero on create = true 2022-11-27T14:34:21.553 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.553 INFO:teuthology.orchestra.run.smithi137.stdout: filestore ondisk finisher threads = 3 2022-11-27T14:34:21.553 INFO:teuthology.orchestra.run.smithi137.stdout: filestore apply finisher threads = 3 2022-11-27T14:34:21.554 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.554 INFO:teuthology.orchestra.run.smithi137.stdout: bdev debug aio = true 2022-11-27T14:34:21.554 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug misdirected ops = true 2022-11-27T14:34:21.554 INFO:teuthology.orchestra.run.smithi137.stdout: bdev async discard = True 2022-11-27T14:34:21.554 INFO:teuthology.orchestra.run.smithi137.stdout: bdev enable discard = True 2022-11-27T14:34:21.555 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore allocator = bitmap 2022-11-27T14:34:21.555 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore block size = 96636764160 2022-11-27T14:34:21.555 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore fsck on mount = True 2022-11-27T14:34:21.555 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluefs = 1/20 2022-11-27T14:34:21.555 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluestore = 1/20 2022-11-27T14:34:21.556 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2022-11-27T14:34:21.556 INFO:teuthology.orchestra.run.smithi137.stdout: debug osd = 20 2022-11-27T14:34:21.556 INFO:teuthology.orchestra.run.smithi137.stdout: debug rocksdb = 4/10 2022-11-27T14:34:21.556 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T14:34:21.557 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd full ratio = 0.9 2022-11-27T14:34:21.557 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd nearfull ratio = 0.8 2022-11-27T14:34:21.557 INFO:teuthology.orchestra.run.smithi137.stdout: osd failsafe full ratio = 0.95 2022-11-27T14:34:21.557 INFO:teuthology.orchestra.run.smithi137.stdout: osd objectstore = bluestore 2022-11-27T14:34:21.557 INFO:teuthology.orchestra.run.smithi137.stdout: osd op complaint time = 180 2022-11-27T14:34:21.558 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.558 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr] 2022-11-27T14:34:21.558 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2022-11-27T14:34:21.558 INFO:teuthology.orchestra.run.smithi137.stdout: debug mgr = 20 2022-11-27T14:34:21.558 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2022-11-27T14:34:21.559 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2022-11-27T14:34:21.559 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2022-11-27T14:34:21.559 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2022-11-27T14:34:21.559 INFO:teuthology.orchestra.run.smithi137.stdout: mgr/telemetry/nag = false 2022-11-27T14:34:21.559 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.560 INFO:teuthology.orchestra.run.smithi137.stdout:[mon] 2022-11-27T14:34:21.560 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2022-11-27T14:34:21.560 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2022-11-27T14:34:21.560 INFO:teuthology.orchestra.run.smithi137.stdout: debug paxos = 20 2022-11-27T14:34:21.561 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2022-11-27T14:34:21.561 INFO:teuthology.orchestra.run.smithi137.stdout: mon data avail warn = 5 2022-11-27T14:34:21.561 INFO:teuthology.orchestra.run.smithi137.stdout: mon mgr mkfs grace = 240 2022-11-27T14:34:21.561 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2022-11-27T14:34:21.561 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd reporter subtree level = osd 2022-11-27T14:34:21.562 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd prime pg temp = true 2022-11-27T14:34:21.562 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2022-11-27T14:34:21.562 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.562 INFO:teuthology.orchestra.run.smithi137.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T14:34:21.562 INFO:teuthology.orchestra.run.smithi137.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T14:34:21.563 INFO:teuthology.orchestra.run.smithi137.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T14:34:21.563 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.563 INFO:teuthology.orchestra.run.smithi137.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T14:34:21.563 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T14:34:21.563 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T14:34:21.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.564 INFO:teuthology.orchestra.run.smithi137.stdout: # 1m isn't quite enough 2022-11-27T14:34:21.564 INFO:teuthology.orchestra.run.smithi137.stdout: mon_down_mkfs_grace = 2m 2022-11-27T14:34:21.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.565 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_filestore_osds = false 2022-11-27T14:34:21.565 INFO:teuthology.orchestra.run.smithi137.stdout: mon op complaint time = 120 2022-11-27T14:34:21.565 INFO:teuthology.orchestra.run.smithi137.stdout: 2022-11-27T14:34:21.565 INFO:teuthology.orchestra.run.smithi137.stdout:[client] 2022-11-27T14:34:21.565 INFO:teuthology.orchestra.run.smithi137.stdout: rgw cache enabled = true 2022-11-27T14:34:21.566 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable ops log = true 2022-11-27T14:34:21.566 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable usage log = true 2022-11-27T14:34:21.566 INFO:teuthology.orchestra.run.smithi137.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T14:34:21.566 INFO:teuthology.orchestra.run.smithi137.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T14:34:21.566 INFO:teuthology.orchestra.run.smithi137.stdout: client mount timeout = 600 2022-11-27T14:34:21.567 INFO:teuthology.orchestra.run.smithi137.stdout: debug client = 20 2022-11-27T14:34:21.567 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2022-11-27T14:34:21.567 INFO:teuthology.orchestra.run.smithi137.stdout: rados mon op timeout = 900 2022-11-27T14:34:21.567 INFO:teuthology.orchestra.run.smithi137.stdout: rados osd op timeout = 900 2022-11-27T14:34:21.568 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.a] 2022-11-27T14:34:21.568 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.b] 2022-11-27T14:34:21.568 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.c] 2022-11-27T14:34:21.568 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.a] 2022-11-27T14:34:21.568 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.c] 2022-11-27T14:34:21.569 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.b] 2022-11-27T14:34:21.569 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.d] 2022-11-27T14:34:21.569 INFO:teuthology.orchestra.run.smithi137.stdout:[mds] 2022-11-27T14:34:21.569 INFO:teuthology.orchestra.run.smithi137.stdout: debug mds = 20 2022-11-27T14:34:21.569 INFO:teuthology.orchestra.run.smithi137.stdout: debug mds balancer = 20 2022-11-27T14:34:21.570 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2022-11-27T14:34:21.570 INFO:teuthology.orchestra.run.smithi137.stdout: mds debug frag = True 2022-11-27T14:34:21.570 INFO:teuthology.orchestra.run.smithi137.stdout: mds debug scatterstat = True 2022-11-27T14:34:21.570 INFO:teuthology.orchestra.run.smithi137.stdout: mds op complaint time = 180 2022-11-27T14:34:21.570 INFO:teuthology.orchestra.run.smithi137.stdout: mds verify scatter = True 2022-11-27T14:34:21.571 INFO:teuthology.orchestra.run.smithi137.stdout: osd op complaint time = 180 2022-11-27T14:34:21.571 INFO:teuthology.orchestra.run.smithi137.stdout: rados mon op timeout = 900 2022-11-27T14:34:21.571 INFO:teuthology.orchestra.run.smithi137.stdout: rados osd op timeout = 900 2022-11-27T14:34:21.572 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-27T14:34:21.572 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-27T14:34:21.635 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-27T14:34:21.635 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:21.636 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-27T14:34:21.653 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:21.653 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-27T14:34:21.709 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2022-11-27T14:34:21.710 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:21.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T14:34:21.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T14:34:21.785 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:21.786 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:21.842 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi137.front.sepia.ceph.com 2022-11-27T14:34:21.843 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:21.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T14:34:21.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T14:34:21.880 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:21.881 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:21.937 INFO:tasks.ceph:Setting up mon nodes... 2022-11-27T14:34:21.937 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-27T14:34:21.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T14:34:21.991 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T14:34:21.993 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T14:34:22.347 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T14:34:22.350 INFO:tasks.ceph:Setting up mds nodes... 2022-11-27T14:34:22.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T14:34:22.405 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T14:34:22.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-27T14:34:22.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T14:34:22.527 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T14:34:22.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-27T14:34:22.555 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T14:34:22.615 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T14:34:22.617 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-27T14:34:22.646 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T14:34:22.740 INFO:teuthology.orchestra.run.smithi137.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T14:34:22.742 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-11-27T14:34:22.768 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-27T14:34:22.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-27T14:34:22.813 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-27T14:34:22.826 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-27T14:34:22.868 INFO:teuthology.orchestra.run.smithi137.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-27T14:34:22.881 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-27T14:34:22.881 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T14:34:22.882 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-27T14:34:22.909 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:22.909 INFO:tasks.ceph:role: osd.0 2022-11-27T14:34:22.909 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2022-11-27T14:34:22.910 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T14:34:22.983 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T14:34:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T14:34:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T14:34:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T14:34:22.984 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T14:34:22.984 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2022-11-27T14:34:22.985 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T14:34:22.985 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T14:34:22.985 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T14:34:22.985 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T14:34:23.419 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2022-11-27T14:34:23.421 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2022-11-27T14:34:23.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-27T14:34:23.588 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-27T14:34:23.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-27T14:34:23.690 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:23.691 INFO:tasks.ceph:role: osd.1 2022-11-27T14:34:23.691 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2022-11-27T14:34:23.691 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T14:34:23.765 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T14:34:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T14:34:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T14:34:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T14:34:23.766 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T14:34:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2022-11-27T14:34:23.766 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T14:34:23.766 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T14:34:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T14:34:23.767 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T14:34:24.191 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2022-11-27T14:34:24.192 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2022-11-27T14:34:24.193 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-11-27T14:34:24.307 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-27T14:34:24.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-27T14:34:24.404 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:24.404 INFO:tasks.ceph:role: osd.2 2022-11-27T14:34:24.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2022-11-27T14:34:24.405 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T14:34:24.477 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T14:34:24.478 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T14:34:24.478 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T14:34:24.478 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T14:34:24.479 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T14:34:24.479 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2022-11-27T14:34:24.479 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T14:34:24.479 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T14:34:24.480 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T14:34:24.480 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T14:34:24.922 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2022-11-27T14:34:24.924 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2022-11-27T14:34:24.924 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-11-27T14:34:25.079 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-27T14:34:25.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-27T14:34:25.180 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:25.180 INFO:tasks.ceph:role: osd.3 2022-11-27T14:34:25.181 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2022-11-27T14:34:25.181 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T14:34:25.254 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T14:34:25.254 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T14:34:25.254 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T14:34:25.255 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T14:34:25.255 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T14:34:25.255 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2022-11-27T14:34:25.255 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T14:34:25.256 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T14:34:25.256 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T14:34:25.257 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T14:34:25.693 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2022-11-27T14:34:25.695 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2022-11-27T14:34:25.696 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-11-27T14:34:25.874 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-27T14:34:25.912 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:26.003 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:26.001+0000 7f7c020ed5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-27T14:34:26.004 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:26.002+0000 7f7c020ed5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-27T14:34:26.004 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:26.002+0000 7f7c020ed5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T14:34:26.004 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:26.002+0000 7f7c020ed5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T14:34:26.004 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:26.002+0000 7f7c020ed5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T14:34:26.005 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:26.002+0000 7f7c020ed5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-11-27T14:34:28.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-27T14:34:28.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:28.990+0000 7fa6d91c15c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-27T14:34:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:28.990+0000 7fa6d91c15c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-27T14:34:28.994 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:28.990+0000 7fa6d91c15c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T14:34:28.994 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:28.990+0000 7fa6d91c15c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T14:34:28.994 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:28.990+0000 7fa6d91c15c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T14:34:28.995 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:28.991+0000 7fa6d91c15c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-11-27T14:34:31.861 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-27T14:34:31.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:31.981 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:31.979+0000 7fe4322cc5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-27T14:34:31.981 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:31.979+0000 7fe4322cc5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-27T14:34:31.982 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:31.979+0000 7fe4322cc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T14:34:31.982 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:31.979+0000 7fe4322cc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T14:34:31.982 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:31.979+0000 7fe4322cc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T14:34:31.982 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:31.980+0000 7fe4322cc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-11-27T14:34:34.843 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-27T14:34:34.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:34.963 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:34.961+0000 7f059a8cc5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-27T14:34:34.964 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:34.961+0000 7f059a8cc5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-27T14:34:34.964 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:34.961+0000 7f059a8cc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T14:34:34.964 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:34.961+0000 7f059a8cc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T14:34:34.965 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:34.961+0000 7f059a8cc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T14:34:34.965 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:34.962+0000 7f059a8cc5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-11-27T14:34:37.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-27T14:34:37.878 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-11-27T14:34:37.912 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:37.913 INFO:tasks.ceph:role: osd.4 2022-11-27T14:34:37.913 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com 2022-11-27T14:34:37.914 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T14:34:37.991 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T14:34:37.992 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T14:34:37.992 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T14:34:37.992 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T14:34:37.993 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T14:34:37.993 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2022-11-27T14:34:37.993 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T14:34:37.993 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T14:34:37.994 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T14:34:37.994 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T14:34:38.442 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2022-11-27T14:34:38.444 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2022-11-27T14:34:38.444 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-11-27T14:34:38.624 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-11-27T14:34:38.662 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-11-27T14:34:38.731 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:38.732 INFO:tasks.ceph:role: osd.5 2022-11-27T14:34:38.732 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com 2022-11-27T14:34:38.733 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T14:34:38.806 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T14:34:38.807 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T14:34:38.807 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T14:34:38.807 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T14:34:38.807 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T14:34:38.808 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2022-11-27T14:34:38.808 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T14:34:38.808 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T14:34:38.808 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T14:34:38.808 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T14:34:39.223 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2022-11-27T14:34:39.224 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2022-11-27T14:34:39.225 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-11-27T14:34:39.343 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-11-27T14:34:39.375 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-11-27T14:34:39.443 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:39.444 INFO:tasks.ceph:role: osd.6 2022-11-27T14:34:39.444 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com 2022-11-27T14:34:39.444 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T14:34:39.518 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T14:34:39.518 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T14:34:39.518 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T14:34:39.518 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T14:34:39.519 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T14:34:39.519 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2022-11-27T14:34:39.519 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T14:34:39.519 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T14:34:39.520 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T14:34:39.520 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T14:34:39.964 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2022-11-27T14:34:39.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2022-11-27T14:34:39.967 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-11-27T14:34:40.108 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-11-27T14:34:40.141 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-11-27T14:34:40.208 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T14:34:40.208 INFO:tasks.ceph:role: osd.7 2022-11-27T14:34:40.208 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com 2022-11-27T14:34:40.209 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T14:34:40.282 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T14:34:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T14:34:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T14:34:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T14:34:40.283 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T14:34:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2022-11-27T14:34:40.284 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T14:34:40.284 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T14:34:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T14:34:40.284 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T14:34:40.725 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2022-11-27T14:34:40.726 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2022-11-27T14:34:40.727 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-11-27T14:34:40.905 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-11-27T14:34:40.938 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:41.026 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:41.025+0000 7f2aa3e7c5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-11-27T14:34:41.026 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:41.025+0000 7f2aa3e7c5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-11-27T14:34:41.026 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:41.025+0000 7f2aa3e7c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T14:34:41.027 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:41.025+0000 7f2aa3e7c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T14:34:41.027 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:41.025+0000 7f2aa3e7c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T14:34:41.027 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:41.025+0000 7f2aa3e7c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-11-27T14:34:43.888 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-11-27T14:34:43.921 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:44.012 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:44.011+0000 7f35e12805c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-11-27T14:34:44.013 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:44.012+0000 7f35e12805c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-11-27T14:34:44.013 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:44.012+0000 7f35e12805c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T14:34:44.014 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:44.012+0000 7f35e12805c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T14:34:44.014 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:44.012+0000 7f35e12805c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T14:34:44.014 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:44.012+0000 7f35e12805c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-11-27T14:34:46.875 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-11-27T14:34:46.907 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:46.998 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:46.997+0000 7f6417bb85c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-11-27T14:34:46.999 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:46.997+0000 7f6417bb85c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-11-27T14:34:46.999 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:46.997+0000 7f6417bb85c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T14:34:46.999 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:46.997+0000 7f6417bb85c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T14:34:47.000 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:46.997+0000 7f6417bb85c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T14:34:47.000 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:46.998+0000 7f6417bb85c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-11-27T14:34:49.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T14:34:49.471 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T14:34:49.863 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-11-27T14:34:49.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:49.982 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:49.981+0000 7ff9924055c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-11-27T14:34:49.983 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:49.982+0000 7ff9924055c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-11-27T14:34:49.983 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:49.982+0000 7ff9924055c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T14:34:49.983 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:49.982+0000 7ff9924055c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T14:34:49.984 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:49.982+0000 7ff9924055c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T14:34:49.984 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:49.982+0000 7ff9924055c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-11-27T14:34:52.842 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-11-27T14:34:52.874 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-27T14:34:52.875 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:52.876 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-27T14:34:52.908 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:52.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-27T14:34:52.978 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:52.979 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-27T14:34:53.047 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:53.048 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-27T14:34:53.115 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:53.115 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-27T14:34:53.183 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:53.184 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-27T14:34:53.251 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:53.251 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-27T14:34:53.318 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:53.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-27T14:34:53.351 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:53.351 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-27T14:34:53.421 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:53.422 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-11-27T14:34:53.491 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:53.491 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-11-27T14:34:53.559 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:53.559 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-11-27T14:34:53.627 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:53.627 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-11-27T14:34:53.694 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:53.695 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-11-27T14:34:53.761 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:53.762 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-27T14:34:53.782 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:53.782 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-27T14:34:53.818 INFO:tasks.ceph:Adding keys to all mons... 2022-11-27T14:34:53.819 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T14:34:53.825 DEBUG:teuthology.orchestra.run.smithi137:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T14:34:53.862 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2022-11-27T14:34:53.862 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBtdYNjBX0ROxAApWfTU+8GV5/+YvAKykuoeg== 2022-11-27T14:34:53.862 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2022-11-27T14:34:53.862 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBudYNjGNEtGBAAUVGIPt26AlBkotSr64xs2w== 2022-11-27T14:34:53.863 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2022-11-27T14:34:53.863 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBudYNjKK9qHxAA8TRLxKMkXgdySQIhZKQ3gQ== 2022-11-27T14:34:53.863 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2022-11-27T14:34:53.863 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBydYNjoVwvABAAj8og4j+4R3CiUTG5QI9XCw== 2022-11-27T14:34:53.864 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2022-11-27T14:34:53.864 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB0dYNjOY4eOxAAuPv8m+4HW8vDPBDiDD1HUg== 2022-11-27T14:34:53.864 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2022-11-27T14:34:53.865 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB3dYNjsPB1OhAAOEHdeyR92lFIJOTOWdagig== 2022-11-27T14:34:53.865 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2022-11-27T14:34:53.865 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQB6dYNjXKVmORAASTpUgxKffuCPP/FCtT/KAw== 2022-11-27T14:34:53.865 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2022-11-27T14:34:53.866 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBudYNj2iKyFBAAdYhx8A5jB9jEA4RB+AmRFw== 2022-11-27T14:34:53.866 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2022-11-27T14:34:53.866 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBudYNjoi+qJBAA5F/IAzN8L5/He98HUDG7hg== 2022-11-27T14:34:53.866 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2022-11-27T14:34:53.867 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBudYNjt6YbLBAAMNz/CnB2vLsKx7YjwD34Ig== 2022-11-27T14:34:53.867 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2022-11-27T14:34:53.867 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQCBdYNj3lWHARAANmjfFJRGViAEYqgPK5P8qg== 2022-11-27T14:34:53.867 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2022-11-27T14:34:53.868 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQCEdYNjek67ABAA4+D+kqhNcm8xSyfKez810g== 2022-11-27T14:34:53.868 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2022-11-27T14:34:53.868 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQCGdYNjLwx7OxAAIwae29bxAQyTcrR91OgUmg== 2022-11-27T14:34:53.868 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2022-11-27T14:34:53.869 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQCJdYNjaXeLOhAAZlRsLUB4yXz/x+APUF/NGw== 2022-11-27T14:34:53.869 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2022-11-27T14:34:53.869 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBudYNj3915MBAANQJ37uN4feQFSNAqAZoXJQ== 2022-11-27T14:34:53.869 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2022-11-27T14:34:53.869 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBudYNjmQ/IMxAAa2v50faDru0M1wS4hoqRcw== 2022-11-27T14:34:53.883 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.y] 2022-11-27T14:34:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBtdYNjBX0ROxAApWfTU+8GV5/+YvAKykuoeg== 2022-11-27T14:34:53.884 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.a] 2022-11-27T14:34:53.884 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBudYNjGNEtGBAAUVGIPt26AlBkotSr64xs2w== 2022-11-27T14:34:53.884 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.c] 2022-11-27T14:34:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBudYNjKK9qHxAA8TRLxKMkXgdySQIhZKQ3gQ== 2022-11-27T14:34:53.885 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.0] 2022-11-27T14:34:53.885 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBydYNjoVwvABAAj8og4j+4R3CiUTG5QI9XCw== 2022-11-27T14:34:53.885 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.1] 2022-11-27T14:34:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB0dYNjOY4eOxAAuPv8m+4HW8vDPBDiDD1HUg== 2022-11-27T14:34:53.886 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.2] 2022-11-27T14:34:53.886 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB3dYNjsPB1OhAAOEHdeyR92lFIJOTOWdagig== 2022-11-27T14:34:53.886 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.3] 2022-11-27T14:34:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB6dYNjXKVmORAASTpUgxKffuCPP/FCtT/KAw== 2022-11-27T14:34:53.887 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.x] 2022-11-27T14:34:53.887 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBudYNj2iKyFBAAdYhx8A5jB9jEA4RB+AmRFw== 2022-11-27T14:34:53.887 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.b] 2022-11-27T14:34:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBudYNjoi+qJBAA5F/IAzN8L5/He98HUDG7hg== 2022-11-27T14:34:53.888 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.d] 2022-11-27T14:34:53.888 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBudYNjt6YbLBAAMNz/CnB2vLsKx7YjwD34Ig== 2022-11-27T14:34:53.888 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.4] 2022-11-27T14:34:53.889 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCBdYNj3lWHARAANmjfFJRGViAEYqgPK5P8qg== 2022-11-27T14:34:53.889 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.5] 2022-11-27T14:34:53.889 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCEdYNjek67ABAA4+D+kqhNcm8xSyfKez810g== 2022-11-27T14:34:53.889 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.6] 2022-11-27T14:34:53.889 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCGdYNjLwx7OxAAIwae29bxAQyTcrR91OgUmg== 2022-11-27T14:34:53.890 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.7] 2022-11-27T14:34:53.890 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQCJdYNjaXeLOhAAZlRsLUB4yXz/x+APUF/NGw== 2022-11-27T14:34:53.890 INFO:teuthology.orchestra.run.smithi137.stdout:[client.0] 2022-11-27T14:34:53.890 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBudYNj3915MBAANQJ37uN4feQFSNAqAZoXJQ== 2022-11-27T14:34:53.891 INFO:teuthology.orchestra.run.smithi137.stdout:[client.1] 2022-11-27T14:34:53.891 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQBudYNjmQ/IMxAAa2v50faDru0M1wS4hoqRcw== 2022-11-27T14:34:53.892 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T14:34:53.914 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T14:34:53.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T14:34:53.989 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T14:34:54.045 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T14:34:54.047 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T14:34:54.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.106 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.160 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.164 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.217 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.220 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.278 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.281 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T14:34:54.336 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T14:34:54.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T14:34:54.391 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T14:34:54.447 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T14:34:54.449 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T14:34:54.505 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.508 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.560 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.563 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.616 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.618 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.671 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.674 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T14:34:54.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T14:34:54.729 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T14:34:54.786 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T14:34:54.789 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T14:34:54.845 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-27T14:34:54.846 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-27T14:34:54.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T14:34:54.970 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-27T14:34:54.997 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-27T14:34:55.025 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T14:34:55.122 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-27T14:34:55.149 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-27T14:34:55.216 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T14:34:55.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-27T14:34:55.341 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:55.343 DEBUG:teuthology.orchestra.run.smithi137:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T14:34:55.398 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-27T14:34:55.399 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-27T14:34:55.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-27T14:34:55.402 INFO:tasks.ceph.mon.a:Started 2022-11-27T14:34:55.402 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-27T14:34:55.403 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-27T14:34:55.441 INFO:tasks.ceph.mon.b:Started 2022-11-27T14:34:55.442 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-27T14:34:55.443 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-27T14:34:55.446 INFO:tasks.ceph.mon.c:Started 2022-11-27T14:34:55.446 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-27T14:34:55.447 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-27T14:34:55.448 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-27T14:34:55.451 INFO:tasks.ceph.mgr.y:Started 2022-11-27T14:34:55.452 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-27T14:34:55.452 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-27T14:34:55.455 INFO:tasks.ceph.mgr.x:Started 2022-11-27T14:34:55.456 INFO:tasks.ceph:Setting crush tunables to default 2022-11-27T14:34:55.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd crush tunables default 2022-11-27T14:34:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.577+0000 7fc2e8eb2700 1 Processor -- start 2022-11-27T14:34:55.579 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.577+0000 7fc2e8eb2700 1 -- start start 2022-11-27T14:34:55.581 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e8eb2700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 0x7fc2e4156140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:55.581 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e8eb2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 0x7fc2e40be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:55.582 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e8eb2700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e40c1bf0 con 0x7fc2e41501a0 2022-11-27T14:34:55.582 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e8eb2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e40c0270 con 0x7fc2e4069be0 2022-11-27T14:34:55.582 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e8eb2700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e4069750 con 0x7fc2e40c1070 2022-11-27T14:34:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e2d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 0x7fc2e40be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e2d9d700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 0x7fc2e40be750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54630/0 (socket says 172.21.15.99:54630) 2022-11-27T14:34:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e2d9d700 1 -- 172.21.15.99:0/264055862 learned_addr learned my addr 172.21.15.99:0/264055862 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:34:55.583 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e1d9b700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 0x7fc2e4156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:55.584 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e2d9d700 1 -- 172.21.15.99:0/264055862 >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 msgr2=0x7fc2e40be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-27T14:34:55.584 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e2d9d700 1 -- 172.21.15.99:0/264055862 >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 msgr2=0x7fc2e40be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T14:34:55.584 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e2d9d700 1 --2- 172.21.15.99:0/264055862 >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 0x7fc2e40be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T14:34:55.584 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e2d9d700 1 --2- 172.21.15.99:0/264055862 >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 0x7fc2e40be750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T14:34:55.585 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e1d9b700 1 -- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 msgr2=0x7fc2e4156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T14:34:55.585 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e1d9b700 1 -- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 msgr2=0x7fc2e4156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T14:34:55.585 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e1d9b700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 0x7fc2e4156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T14:34:55.586 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.578+0000 7fc2e1d9b700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 0x7fc2e4156140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T14:34:55.753 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.751+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713020837 0 0) 0x7fc2e40c1bf0 con 0x7fc2e41501a0 2022-11-27T14:34:55.753 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.751+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c80035b0 con 0x7fc2e41501a0 2022-11-27T14:34:55.763 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 <== mon.1 v1:172.21.15.137:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4276471519 0 0) 0x7fc2c80035b0 con 0x7fc2e41501a0 2022-11-27T14:34:55.764 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 msgr2=0x7fc2e4156140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T14:34:55.764 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 0x7fc2e4156140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.764 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 msgr2=0x7fc2e40be750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T14:34:55.764 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 --2- 172.21.15.99:0/264055862 >> v2:172.21.15.99:3300/0 conn(0x7fc2e4069be0 0x7fc2e40be750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.765 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 --> v1:172.21.15.137:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e4069420 con 0x7fc2e41501a0 2022-11-27T14:34:55.765 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 <== mon.1 v1:172.21.15.137:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2287561895 0 0) 0x7fc2d8001670 con 0x7fc2e41501a0 2022-11-27T14:34:55.765 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 >> v1:172.21.15.137:6789/0 conn(0x7fc2e41501a0 legacy=0x7fc2e406a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:55.765 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8003a00 0x7fc2c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:55.766 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 0x7fc2c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:55.766 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c8004d90 0x7fc2c8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:55.766 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e40c0270 con 0x7fc2c8004d90 2022-11-27T14:34:55.766 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e4069750 con 0x7fc2c8003a00 2022-11-27T14:34:55.767 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.756+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c800baf0 con 0x7fc2e40c1070 2022-11-27T14:34:55.767 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e2d9d700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c8004d90 0x7fc2c8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:55.767 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e259c700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8003a00 0x7fc2c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:55.767 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e1d9b700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 0x7fc2c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:55.768 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e2d9d700 1 -- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 msgr2=0x7fc2c8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:55.768 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e2d9d700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c1070 0x7fc2c8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.768 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e2d9d700 1 -- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8003a00 msgr2=0x7fc2c8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:55.768 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e2d9d700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8003a00 0x7fc2c8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.769 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e2d9d700 1 -- 172.21.15.99:0/264055862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80035b0 con 0x7fc2c8004d90 2022-11-27T14:34:55.769 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e2d9d700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c8004d90 0x7fc2c8005200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009cc0 tx=0x7fc2c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95da4e86874beba2 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:55.769 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc017070 con 0x7fc2c8004d90 2022-11-27T14:34:55.769 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc00f040 con 0x7fc2c8004d90 2022-11-27T14:34:55.769 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.757+0000 7fc2e159a700 1 -- 172.21.15.99:0/264055862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc003c60 con 0x7fc2c8004d90 2022-11-27T14:34:55.770 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.758+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/264055862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c8004d90 msgr2=0x7fc2c8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:55.770 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.758+0000 7fc2e8eb2700 1 --2- 172.21.15.99:0/264055862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c8004d90 0x7fc2c8005200 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009cc0 tx=0x7fc2c8005880 comp rx=0 tx=0).stop 2022-11-27T14:34:55.770 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.758+0000 7fc2e2d9d700 1 -- 172.21.15.99:0/264055862 reap_dead start 2022-11-27T14:34:55.770 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.758+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/264055862 shutdown_connections 2022-11-27T14:34:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.758+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/264055862 >> 172.21.15.99:0/264055862 conn(0x7fc2e40c5320 msgr2=0x7fc2e40c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.758+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/264055862 shutdown_connections 2022-11-27T14:34:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.758+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/264055862 wait complete. 2022-11-27T14:34:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.759+0000 7fc2e8eb2700 1 Processor -- start 2022-11-27T14:34:55.771 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.759+0000 7fc2e8eb2700 1 -- start start 2022-11-27T14:34:55.772 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e8eb2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8004d90 0x7fc2e41ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:55.772 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e8eb2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e40aa600 0x7fc2e41e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:55.772 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e8eb2700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c4990 0x7fc2e4154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:55.772 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e8eb2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e41e89b0 con 0x7fc2e40aa600 2022-11-27T14:34:55.773 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e8eb2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e4150dc0 con 0x7fc2c8004d90 2022-11-27T14:34:55.773 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e8eb2700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e40a9ad0 con 0x7fc2e40c4990 2022-11-27T14:34:55.773 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e40aa600 0x7fc2e41e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:55.773 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e1d9b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e40aa600 0x7fc2e41e8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54658/0 (socket says 172.21.15.99:54658) 2022-11-27T14:34:55.774 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e259c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8004d90 0x7fc2e41ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:55.774 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e1d9b700 1 -- 172.21.15.99:0/2470390675 learned_addr learned my addr 172.21.15.99:0/2470390675 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:34:55.774 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e2d9d700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c4990 0x7fc2e4154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:55.774 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e1d9b700 1 -- 172.21.15.99:0/2470390675 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c4990 msgr2=0x7fc2e4154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:55.775 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e1d9b700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c4990 0x7fc2e4154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.775 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e1d9b700 1 -- 172.21.15.99:0/2470390675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8004d90 msgr2=0x7fc2e41ec2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:55.775 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e1d9b700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8004d90 0x7fc2e41ec2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.775 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e1d9b700 1 -- 172.21.15.99:0/2470390675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e41e9310 con 0x7fc2e40aa600 2022-11-27T14:34:55.776 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.760+0000 7fc2e259c700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8004d90 0x7fc2e41ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:55.776 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.761+0000 7fc2e1d9b700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e40aa600 0x7fc2e41e8dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4010a20 tx=0x7fc2d4010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:55.776 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.761+0000 7fc2d2ffd700 1 -- 172.21.15.99:0/2470390675 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4006ea0 con 0x7fc2e40aa600 2022-11-27T14:34:55.776 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.761+0000 7fc2d2ffd700 1 -- 172.21.15.99:0/2470390675 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d400ea70 con 0x7fc2e40aa600 2022-11-27T14:34:55.776 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.761+0000 7fc2d2ffd700 1 -- 172.21.15.99:0/2470390675 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4009730 con 0x7fc2e40aa600 2022-11-27T14:34:55.777 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.761+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e41e8350 con 0x7fc2e40aa600 2022-11-27T14:34:55.777 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.761+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e41543a0 con 0x7fc2e40aa600 2022-11-27T14:34:55.777 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.761+0000 7fc2d2ffd700 1 -- 172.21.15.99:0/2470390675 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc2d4026070 con 0x7fc2e40aa600 2022-11-27T14:34:55.777 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.761+0000 7fc2d2ffd700 1 -- 172.21.15.99:0/2470390675 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc2d4015d20 con 0x7fc2e40aa600 2022-11-27T14:34:55.777 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.762+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e41545c0 con 0x7fc2e40aa600 2022-11-27T14:34:55.778 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.764+0000 7fc2d2ffd700 1 -- 172.21.15.99:0/2470390675 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fc2e41e9310 con 0x7fc2e40aa600 2022-11-27T14:34:55.839 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.838+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fc2e40733f0 con 0x7fc2e40aa600 2022-11-27T14:34:55.846 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.844+0000 7fc2d2ffd700 1 -- 172.21.15.99:0/2470390675 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc2d402e090 con 0x7fc2e40aa600 2022-11-27T14:34:55.847 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.845+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e40aa600 msgr2=0x7fc2e41e8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:55.848 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.845+0000 7fc2e8eb2700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e40aa600 0x7fc2e41e8dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4010a20 tx=0x7fc2d4010d30 comp rx=0 tx=0).stop 2022-11-27T14:34:55.848 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.845+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 shutdown_connections 2022-11-27T14:34:55.848 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.845+0000 7fc2e8eb2700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fc2e40c4990 0x7fc2e4154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.849 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.845+0000 7fc2e8eb2700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2c8004d90 0x7fc2e41ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.849 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.845+0000 7fc2e8eb2700 1 --2- 172.21.15.99:0/2470390675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2e40aa600 0x7fc2e41e8dd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:55.849 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.845+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 >> 172.21.15.99:0/2470390675 conn(0x7fc2e40c5320 msgr2=0x7fc2e414f310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:55.849 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.846+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 shutdown_connections 2022-11-27T14:34:55.849 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:55.846+0000 7fc2e8eb2700 1 -- 172.21.15.99:0/2470390675 wait complete. 2022-11-27T14:34:55.850 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2022-11-27T14:34:55.859 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-27T14:34:55.860 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:55.860 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-27T14:34:55.870 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:55.869+0000 7f9b369a1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T14:34:55.886 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:55.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-27T14:34:55.937 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:55.936+0000 7f9b369a1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T14:34:55.952 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:55.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-27T14:34:56.015 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:56.013+0000 7f9b369a1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T14:34:56.018 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2022-11-27T14:34:56.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-27T14:34:56.084 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:56.085 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-11-27T14:34:56.096 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:56.095+0000 7f9b369a1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T14:34:56.115 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:56.115 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-11-27T14:34:56.183 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:56.183 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-11-27T14:34:56.236 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:56.235+0000 7f9b369a1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T14:34:56.249 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2022-11-27T14:34:56.250 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-11-27T14:34:56.251 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:56.250+0000 7fb8f84b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T14:34:56.276 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c 0 2022-11-27T14:34:56.318 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:56.317+0000 7fb8f84b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T14:34:56.396 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:56.395+0000 7fb8f84b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T14:34:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.437+0000 7f2455e84700 1 Processor -- start 2022-11-27T14:34:56.446 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.437+0000 7f2455e84700 1 -- start start 2022-11-27T14:34:56.447 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f2455e84700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f245014dee0 0x7f245006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.447 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f2455e84700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f245006d5f0 0x7f2450069380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.447 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f2455e84700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2450069fd0 con 0x7f245006c210 2022-11-27T14:34:56.448 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f2455e84700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2450065200 con 0x7f245006d5f0 2022-11-27T14:34:56.448 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f2455e84700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24500c00b0 con 0x7f245014dee0 2022-11-27T14:34:56.448 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244f7fe700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f245014dee0 0x7f245006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.449 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244f7fe700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f245014dee0 0x7f245006d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46472/0 (socket says 172.21.15.137:46472) 2022-11-27T14:34:56.449 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244f7fe700 1 -- 172.21.15.137:0/2664545670 learned_addr learned my addr 172.21.15.137:0/2664545670 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:56.450 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244ffff700 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f245006c210 0x7f24500698c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:48548/0 (socket says 172.21.15.137:48548) 2022-11-27T14:34:56.450 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244effd700 1 --2- 172.21.15.137:0/2664545670 >> v2:172.21.15.99:3300/0 conn(0x7f245006d5f0 0x7f2450069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.450 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244e7fc700 1 -- 172.21.15.137:0/2664545670 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643838776 0 0) 0x7f2450069fd0 con 0x7f245006c210 2022-11-27T14:34:56.451 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244e7fc700 1 -- 172.21.15.137:0/2664545670 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f243c0035b0 con 0x7f245006c210 2022-11-27T14:34:56.451 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244f7fe700 1 -- 172.21.15.137:0/2664545670 >> v2:172.21.15.99:3300/0 conn(0x7f245006d5f0 msgr2=0x7f2450069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:56.452 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244f7fe700 1 --2- 172.21.15.137:0/2664545670 >> v2:172.21.15.99:3300/0 conn(0x7f245006d5f0 0x7f2450069380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.452 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244f7fe700 1 -- 172.21.15.137:0/2664545670 >> v1:172.21.15.137:6789/0 conn(0x7f245006c210 legacy=0x7f24500698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.453 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244f7fe700 1 -- 172.21.15.137:0/2664545670 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24501512c0 con 0x7f245014dee0 2022-11-27T14:34:56.453 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.438+0000 7f244f7fe700 1 --2- 172.21.15.137:0/2664545670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 0x7f245006d0b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24500c5280 tx=0x7f2440009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb44bb20cdbb247 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:56.454 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f244e7fc700 1 -- 172.21.15.137:0/2664545670 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440011830 con 0x7f245014dee0 2022-11-27T14:34:56.454 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f244e7fc700 1 -- 172.21.15.137:0/2664545670 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2440011990 con 0x7f245014dee0 2022-11-27T14:34:56.454 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f244e7fc700 1 -- 172.21.15.137:0/2664545670 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440011d20 con 0x7f245014dee0 2022-11-27T14:34:56.455 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f2455e84700 1 -- 172.21.15.137:0/2664545670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 msgr2=0x7f245006d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.455 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f2455e84700 1 --2- 172.21.15.137:0/2664545670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 0x7f245006d0b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f24500c5280 tx=0x7f2440009bd0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.456 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f2455e84700 1 -- 172.21.15.137:0/2664545670 shutdown_connections 2022-11-27T14:34:56.456 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f2455e84700 1 --2- 172.21.15.137:0/2664545670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 0x7f245006d0b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.457 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f2455e84700 1 --2- 172.21.15.137:0/2664545670 >> v2:172.21.15.99:3300/0 conn(0x7f245006d5f0 0x7f2450069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.457 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f2455e84700 1 -- 172.21.15.137:0/2664545670 >> 172.21.15.137:0/2664545670 conn(0x7f24500c5320 msgr2=0x7f24500c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:56.458 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f2455e84700 1 -- 172.21.15.137:0/2664545670 shutdown_connections 2022-11-27T14:34:56.458 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.439+0000 7f2455e84700 1 -- 172.21.15.137:0/2664545670 wait complete. 2022-11-27T14:34:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.440+0000 7f2455e84700 1 Processor -- start 2022-11-27T14:34:56.459 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f2455e84700 1 -- start start 2022-11-27T14:34:56.460 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f2455e84700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f245006c210 0x7f245014f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.460 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f2455e84700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f245006d5f0 0x7f24501e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f2455e84700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 0x7f2450146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f2455e84700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450147bb0 con 0x7f245006c210 2022-11-27T14:34:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f2455e84700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245014f110 con 0x7f245014dee0 2022-11-27T14:34:56.462 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f2455e84700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24501e9bb0 con 0x7f245006d5f0 2022-11-27T14:34:56.462 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f244ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 0x7f2450146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.463 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f244effd700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f245006d5f0 0x7f24501e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.463 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f244effd700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f245006d5f0 0x7f24501e8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:42502/0 (socket says 172.21.15.137:42502) 2022-11-27T14:34:56.464 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.441+0000 7f244effd700 1 -- 172.21.15.137:0/4091506471 learned_addr learned my addr 172.21.15.137:0/4091506471 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:56.464 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f244f7fe700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f245006c210 0x7f245014f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.465 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f244effd700 1 -- 172.21.15.137:0/4091506471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 msgr2=0x7f2450146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.465 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f244effd700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 0x7f2450146a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.465 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f244effd700 1 -- 172.21.15.137:0/4091506471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f245006c210 msgr2=0x7f245014f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.466 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f244effd700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f245006c210 0x7f245014f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.466 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f244effd700 1 -- 172.21.15.137:0/4091506471 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2440009550 con 0x7f245006d5f0 2022-11-27T14:34:56.467 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f244ffff700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 0x7f2450146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:56.467 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f244effd700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f245006d5f0 0x7f24501e8ee0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f244400e810 tx=0x7f244400eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:56.467 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f2437fff700 1 -- 172.21.15.137:0/4091506471 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444013610 con 0x7f245006d5f0 2022-11-27T14:34:56.468 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f2437fff700 1 -- 172.21.15.137:0/4091506471 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2444013770 con 0x7f245006d5f0 2022-11-27T14:34:56.468 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2450150580 con 0x7f245006d5f0 2022-11-27T14:34:56.468 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f2437fff700 1 -- 172.21.15.137:0/4091506471 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444013b20 con 0x7f245006d5f0 2022-11-27T14:34:56.469 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2450151d80 con 0x7f245006d5f0 2022-11-27T14:34:56.469 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f2437fff700 1 -- 172.21.15.137:0/4091506471 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f2444023070 con 0x7f245006d5f0 2022-11-27T14:34:56.470 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.442+0000 7f2437fff700 1 -- 172.21.15.137:0/4091506471 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2444012370 con 0x7f245006d5f0 2022-11-27T14:34:56.470 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.443+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24500c1760 con 0x7f245006d5f0 2022-11-27T14:34:56.471 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.447+0000 7f2437fff700 1 -- 172.21.15.137:0/4091506471 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f244403b6d0 con 0x7f245006d5f0 2022-11-27T14:34:56.480 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:56.480+0000 7fb8f84b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T14:34:56.532 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.531+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c", "id": 0} v 0) v1 -- 0x7f24500733f0 con 0x7f245006d5f0 2022-11-27T14:34:56.537 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.537+0000 7f2437fff700 1 -- 172.21.15.137:0/4091506471 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f244402a030 con 0x7f245006d5f0 2022-11-27T14:34:56.538 INFO:teuthology.orchestra.run.smithi137.stdout:0 2022-11-27T14:34:56.540 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f245006d5f0 msgr2=0x7f24501e8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.540 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f245006d5f0 0x7f24501e8ee0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f244400e810 tx=0x7f244400eb20 comp rx=0 tx=0).stop 2022-11-27T14:34:56.541 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 shutdown_connections 2022-11-27T14:34:56.541 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f245014dee0 0x7f2450146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.542 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f245006c210 0x7f245014f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.542 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 --2- 172.21.15.137:0/4091506471 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f245006d5f0 0x7f24501e8ee0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.543 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 >> 172.21.15.137:0/4091506471 conn(0x7f24500c5320 msgr2=0x7f24500c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:56.543 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 shutdown_connections 2022-11-27T14:34:56.543 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.538+0000 7f2455e84700 1 -- 172.21.15.137:0/4091506471 wait complete. 2022-11-27T14:34:56.550 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 6cb243f2-dcee-4f20-87da-02657be3dce8 1 2022-11-27T14:34:56.558 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:56.557+0000 7f9b369a1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T14:34:56.627 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:56.626+0000 7fb8f84b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T14:34:56.631 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:56.630+0000 7f9b369a1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T14:34:56.671 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.670+0000 7f8dfa2e2700 1 Processor -- start 2022-11-27T14:34:56.671 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.670+0000 7f8dfa2e2700 1 -- start start 2022-11-27T14:34:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8dfa2e2700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8dfa2e2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8df4156140 0x7f8df41565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8dfa2e2700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df414cf70 con 0x7f8df414cba0 2022-11-27T14:34:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8dfa2e2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df40c0630 con 0x7f8df4156140 2022-11-27T14:34:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8dfa2e2700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df4153a30 con 0x7f8df40bf820 2022-11-27T14:34:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8df37fe700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8df3fff700 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f8df414cba0 0x7f8df4069380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:48556/0 (socket says 172.21.15.137:48556) 2022-11-27T14:34:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8df3fff700 1 -- 172.21.15.137:0/3121995242 learned_addr learned my addr 172.21.15.137:0/3121995242 (peer_addr_for_me v1:172.21.15.137:0/0) 2022-11-27T14:34:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.671+0000 7f8df887f700 1 --2- 172.21.15.137:0/3121995242 >> v2:172.21.15.99:3300/0 conn(0x7f8df4156140 0x7f8df41565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df2ffd700 1 -- 172.21.15.137:0/3121995242 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045691093 0 0) 0x7f8df414cf70 con 0x7f8df414cba0 2022-11-27T14:34:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df37fe700 1 -- 172.21.15.137:0/3121995242 >> v2:172.21.15.99:3300/0 conn(0x7f8df4156140 msgr2=0x7f8df41565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df37fe700 1 --2- 172.21.15.137:0/3121995242 >> v2:172.21.15.99:3300/0 conn(0x7f8df4156140 0x7f8df41565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df37fe700 1 -- 172.21.15.137:0/3121995242 >> v1:172.21.15.137:6789/0 conn(0x7f8df414cba0 legacy=0x7f8df4069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df37fe700 1 -- 172.21.15.137:0/3121995242 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40a4f20 con 0x7f8df40bf820 2022-11-27T14:34:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df37fe700 1 --2- 172.21.15.137:0/3121995242 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41534f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009840 tx=0x7f8de8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637b8306f5c22350 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df2ffd700 1 -- 172.21.15.137:0/3121995242 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8011830 con 0x7f8df40bf820 2022-11-27T14:34:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df2ffd700 1 -- 172.21.15.137:0/3121995242 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de8011990 con 0x7f8df40bf820 2022-11-27T14:34:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8df2ffd700 1 -- 172.21.15.137:0/3121995242 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8011d20 con 0x7f8df40bf820 2022-11-27T14:34:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3121995242 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 msgr2=0x7f8df41534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.679 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8dfa2e2700 1 --2- 172.21.15.137:0/3121995242 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41534f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009840 tx=0x7f8de8009b50 comp rx=0 tx=0).stop 2022-11-27T14:34:56.679 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3121995242 shutdown_connections 2022-11-27T14:34:56.679 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8dfa2e2700 1 --2- 172.21.15.137:0/3121995242 >> v2:172.21.15.99:3300/0 conn(0x7f8df4156140 0x7f8df41565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.679 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8dfa2e2700 1 --2- 172.21.15.137:0/3121995242 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41534f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.679 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.672+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3121995242 >> 172.21.15.137:0/3121995242 conn(0x7f8df40a77f0 msgr2=0x7f8df40a7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:56.680 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.673+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3121995242 shutdown_connections 2022-11-27T14:34:56.680 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.673+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3121995242 wait complete. 2022-11-27T14:34:56.680 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.674+0000 7f8dfa2e2700 1 Processor -- start 2022-11-27T14:34:56.683 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.674+0000 7f8dfa2e2700 1 -- start start 2022-11-27T14:34:56.683 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8dfa2e2700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41ebad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.683 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8dfa2e2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df414cba0 0x7f8df40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8dfa2e2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8df4156140 0x7f8df4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8dfa2e2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41eb360 con 0x7f8df4156140 2022-11-27T14:34:56.684 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8dfa2e2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41dd230 con 0x7f8df414cba0 2022-11-27T14:34:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8dfa2e2700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41dd510 con 0x7f8df40bf820 2022-11-27T14:34:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df3fff700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41ebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df3fff700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41ebad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:42512/0 (socket says 172.21.15.137:42512) 2022-11-27T14:34:56.685 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df414cba0 0x7f8df40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df3fff700 1 -- 172.21.15.137:0/3651472637 learned_addr learned my addr 172.21.15.137:0/3651472637 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df887f700 1 --2- 172.21.15.137:0/3651472637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8df4156140 0x7f8df4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df3fff700 1 -- 172.21.15.137:0/3651472637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df414cba0 msgr2=0x7f8df40694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df3fff700 1 --2- 172.21.15.137:0/3651472637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df414cba0 0x7f8df40694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.686 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df3fff700 1 -- 172.21.15.137:0/3651472637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8df4156140 msgr2=0x7f8df4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.687 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df3fff700 1 --2- 172.21.15.137:0/3651472637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8df4156140 0x7f8df4070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.687 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.675+0000 7f8df3fff700 1 -- 172.21.15.137:0/3651472637 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80094a0 con 0x7f8df40bf820 2022-11-27T14:34:56.687 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.676+0000 7f8df3fff700 1 --2- 172.21.15.137:0/3651472637 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41ebad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8de40011e0 tx=0x7f8de400ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:56.687 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.676+0000 7f8df0ff9700 1 -- 172.21.15.137:0/3651472637 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4004c80 con 0x7f8df40bf820 2022-11-27T14:34:56.688 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.676+0000 7f8df0ff9700 1 -- 172.21.15.137:0/3651472637 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de4004e00 con 0x7f8df40bf820 2022-11-27T14:34:56.688 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.676+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df41dd7f0 con 0x7f8df40bf820 2022-11-27T14:34:56.688 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.676+0000 7f8df0ff9700 1 -- 172.21.15.137:0/3651472637 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4007310 con 0x7f8df40bf820 2022-11-27T14:34:56.688 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.676+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df4147570 con 0x7f8df40bf820 2022-11-27T14:34:56.689 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.676+0000 7f8df0ff9700 1 -- 172.21.15.137:0/3651472637 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8de401a020 con 0x7f8df40bf820 2022-11-27T14:34:56.689 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.676+0000 7f8df0ff9700 1 -- 172.21.15.137:0/3651472637 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f8de401dde0 con 0x7f8df40bf820 2022-11-27T14:34:56.689 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.677+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df41eb7f0 con 0x7f8df40bf820 2022-11-27T14:34:56.689 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.680+0000 7f8df0ff9700 1 -- 172.21.15.137:0/3651472637 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f8df41eb7f0 con 0x7f8df40bf820 2022-11-27T14:34:56.760 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:56.758+0000 7f9b369a1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T14:34:56.763 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.763+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6cb243f2-dcee-4f20-87da-02657be3dce8", "id": 1} v 0) v1 -- 0x7f8df41dd980 con 0x7f8df40bf820 2022-11-27T14:34:56.769 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.768+0000 7f8df0ff9700 1 -- 172.21.15.137:0/3651472637 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6cb243f2-dcee-4f20-87da-02657be3dce8", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8de4007470 con 0x7f8df40bf820 2022-11-27T14:34:56.769 INFO:teuthology.orchestra.run.smithi137.stdout:1 2022-11-27T14:34:56.770 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.769+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 msgr2=0x7f8df41ebad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.771 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.769+0000 7f8dfa2e2700 1 --2- 172.21.15.137:0/3651472637 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41ebad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8de40011e0 tx=0x7f8de400ca30 comp rx=0 tx=0).stop 2022-11-27T14:34:56.771 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.769+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 shutdown_connections 2022-11-27T14:34:56.771 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.769+0000 7f8dfa2e2700 1 --2- 172.21.15.137:0/3651472637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8df4156140 0x7f8df4070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.772 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.769+0000 7f8dfa2e2700 1 --2- 172.21.15.137:0/3651472637 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8df40bf820 0x7f8df41ebad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.772 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.769+0000 7f8dfa2e2700 1 --2- 172.21.15.137:0/3651472637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8df414cba0 0x7f8df40694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.772 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.769+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 >> 172.21.15.137:0/3651472637 conn(0x7f8df40a77f0 msgr2=0x7f8df40c4920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:56.772 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.770+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 shutdown_connections 2022-11-27T14:34:56.773 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.770+0000 7f8dfa2e2700 1 -- 172.21.15.137:0/3651472637 wait complete. 2022-11-27T14:34:56.782 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 6206b34c-8c3d-4337-9f81-27efd1811bd4 2 2022-11-27T14:34:56.839 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:56.837+0000 7f9b369a1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T14:34:56.901 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.901+0000 7ffb006a2700 1 Processor -- start 2022-11-27T14:34:56.902 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.901+0000 7ffb006a2700 1 -- start start 2022-11-27T14:34:56.905 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffb006a2700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ffaf806c210 0x7ffaf8069380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.905 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffb006a2700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ffaf80698c0 0x7ffaf815b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.905 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffb006a2700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf80c00b0 con 0x7ffaf814dee0 2022-11-27T14:34:56.906 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffb006a2700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf8065200 con 0x7ffaf80698c0 2022-11-27T14:34:56.906 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffb006a2700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf81512c0 con 0x7ffaf806c210 2022-11-27T14:34:56.906 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafdc3d700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ffaf806c210 0x7ffaf8069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.906 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafe43e700 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7ffaf814dee0 0x7ffaf806d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:48568/0 (socket says 172.21.15.137:48568) 2022-11-27T14:34:56.906 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafe43e700 1 -- 172.21.15.137:0/1624082177 learned_addr learned my addr 172.21.15.137:0/1624082177 (peer_addr_for_me v1:172.21.15.137:0/0) 2022-11-27T14:34:56.907 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafdc3d700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ffaf806c210 0x7ffaf8069380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46498/0 (socket says 172.21.15.137:46498) 2022-11-27T14:34:56.907 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafec3f700 1 --2- 172.21.15.137:0/1624082177 >> v2:172.21.15.99:3300/0 conn(0x7ffaf80698c0 0x7ffaf815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.907 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafd43c700 1 -- 172.21.15.137:0/1624082177 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667682140 0 0) 0x7ffaf80c00b0 con 0x7ffaf814dee0 2022-11-27T14:34:56.907 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafd43c700 1 -- 172.21.15.137:0/1624082177 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae00035b0 con 0x7ffaf814dee0 2022-11-27T14:34:56.908 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafdc3d700 1 -- 172.21.15.137:0/1624082177 >> v2:172.21.15.99:3300/0 conn(0x7ffaf80698c0 msgr2=0x7ffaf815b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:56.908 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafdc3d700 1 --2- 172.21.15.137:0/1624082177 >> v2:172.21.15.99:3300/0 conn(0x7ffaf80698c0 0x7ffaf815b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.908 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafdc3d700 1 -- 172.21.15.137:0/1624082177 >> v1:172.21.15.137:6789/0 conn(0x7ffaf814dee0 legacy=0x7ffaf806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.908 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafdc3d700 1 -- 172.21.15.137:0/1624082177 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf81515a0 con 0x7ffaf806c210 2022-11-27T14:34:56.909 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafdc3d700 1 --2- 172.21.15.137:0/1624082177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf806c210 0x7ffaf8069380 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffaf806ae00 tx=0x7ffae8009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a78d083b99fd8cc1 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:56.909 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafd43c700 1 -- 172.21.15.137:0/1624082177 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011830 con 0x7ffaf806c210 2022-11-27T14:34:56.909 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafd43c700 1 -- 172.21.15.137:0/1624082177 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae8011990 con 0x7ffaf806c210 2022-11-27T14:34:56.909 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.902+0000 7ffafd43c700 1 -- 172.21.15.137:0/1624082177 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011d20 con 0x7ffaf806c210 2022-11-27T14:34:56.909 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.903+0000 7ffb006a2700 1 -- 172.21.15.137:0/1624082177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf806c210 msgr2=0x7ffaf8069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.910 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.903+0000 7ffb006a2700 1 --2- 172.21.15.137:0/1624082177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf806c210 0x7ffaf8069380 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffaf806ae00 tx=0x7ffae8009b20 comp rx=0 tx=0).stop 2022-11-27T14:34:56.910 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.903+0000 7ffb006a2700 1 -- 172.21.15.137:0/1624082177 shutdown_connections 2022-11-27T14:34:56.910 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.903+0000 7ffb006a2700 1 --2- 172.21.15.137:0/1624082177 >> v2:172.21.15.99:3300/0 conn(0x7ffaf80698c0 0x7ffaf815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.910 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.903+0000 7ffb006a2700 1 --2- 172.21.15.137:0/1624082177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf806c210 0x7ffaf8069380 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.911 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.903+0000 7ffb006a2700 1 -- 172.21.15.137:0/1624082177 >> 172.21.15.137:0/1624082177 conn(0x7ffaf80c5320 msgr2=0x7ffaf80c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:56.911 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.903+0000 7ffb006a2700 1 -- 172.21.15.137:0/1624082177 shutdown_connections 2022-11-27T14:34:56.911 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.903+0000 7ffb006a2700 1 -- 172.21.15.137:0/1624082177 wait complete. 2022-11-27T14:34:56.911 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.904+0000 7ffb006a2700 1 Processor -- start 2022-11-27T14:34:56.912 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.904+0000 7ffb006a2700 1 -- start start 2022-11-27T14:34:56.914 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffb006a2700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffaf80698c0 0x7ffaf81e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.914 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffb006a2700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf806c210 0x7ffaf814c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.915 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffb006a2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf814dee0 0x7ffaf8142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:56.915 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffb006a2700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf814ff30 con 0x7ffaf806c210 2022-11-27T14:34:56.915 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffb006a2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf8143760 con 0x7ffaf814dee0 2022-11-27T14:34:56.915 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffb006a2700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf806f5b0 con 0x7ffaf80698c0 2022-11-27T14:34:56.915 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafec3f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf814dee0 0x7ffaf8142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.916 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafe43e700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffaf80698c0 0x7ffaf81e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.916 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafec3f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf814dee0 0x7ffaf8142c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46504/0 (socket says 172.21.15.137:46504) 2022-11-27T14:34:56.916 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafec3f700 1 -- 172.21.15.137:0/96253382 learned_addr learned my addr 172.21.15.137:0/96253382 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:56.916 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafdc3d700 1 --2- 172.21.15.137:0/96253382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf806c210 0x7ffaf814c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:56.917 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafec3f700 1 -- 172.21.15.137:0/96253382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffaf80698c0 msgr2=0x7ffaf81e63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.917 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafec3f700 1 --2- 172.21.15.137:0/96253382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffaf80698c0 0x7ffaf81e63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.917 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafec3f700 1 -- 172.21.15.137:0/96253382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf806c210 msgr2=0x7ffaf814c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:56.917 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafec3f700 1 --2- 172.21.15.137:0/96253382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf806c210 0x7ffaf814c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:56.918 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.905+0000 7ffafec3f700 1 -- 172.21.15.137:0/96253382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80094a0 con 0x7ffaf814dee0 2022-11-27T14:34:56.918 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.906+0000 7ffafec3f700 1 --2- 172.21.15.137:0/96253382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf814dee0 0x7ffaf8142c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000efc0 tx=0x7ffaf000c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:56.918 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.906+0000 7ffae6ffd700 1 -- 172.21.15.137:0/96253382 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf000cad0 con 0x7ffaf814dee0 2022-11-27T14:34:56.919 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.906+0000 7ffae6ffd700 1 -- 172.21.15.137:0/96253382 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf000f040 con 0x7ffaf814dee0 2022-11-27T14:34:56.919 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.906+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf81e6910 con 0x7ffaf814dee0 2022-11-27T14:34:56.919 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.906+0000 7ffae6ffd700 1 -- 172.21.15.137:0/96253382 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf000ce70 con 0x7ffaf814dee0 2022-11-27T14:34:56.919 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.906+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf8071280 con 0x7ffaf814dee0 2022-11-27T14:34:56.920 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.906+0000 7ffae6ffd700 1 -- 172.21.15.137:0/96253382 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ffaf0014480 con 0x7ffaf814dee0 2022-11-27T14:34:56.920 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.906+0000 7ffae6ffd700 1 -- 172.21.15.137:0/96253382 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7ffaf0013800 con 0x7ffaf814dee0 2022-11-27T14:34:56.920 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.907+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf80c06b0 con 0x7ffaf814dee0 2022-11-27T14:34:56.920 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.909+0000 7ffae6ffd700 1 -- 172.21.15.137:0/96253382 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7ffaf0018020 con 0x7ffaf814dee0 2022-11-27T14:34:56.968 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:56.967+0000 7fb8f84b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T14:34:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.989+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6206b34c-8c3d-4337-9f81-27efd1811bd4", "id": 2} v 0) v1 -- 0x7ffaf80c1760 con 0x7ffaf814dee0 2022-11-27T14:34:56.998 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.997+0000 7ffae6ffd700 1 -- 172.21.15.137:0/96253382 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6206b34c-8c3d-4337-9f81-27efd1811bd4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffaf0013560 con 0x7ffaf814dee0 2022-11-27T14:34:56.998 INFO:teuthology.orchestra.run.smithi137.stdout:2 2022-11-27T14:34:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf814dee0 msgr2=0x7ffaf8142c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 --2- 172.21.15.137:0/96253382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf814dee0 0x7ffaf8142c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000efc0 tx=0x7ffaf000c0c0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 shutdown_connections 2022-11-27T14:34:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 --2- 172.21.15.137:0/96253382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffaf814dee0 0x7ffaf8142c30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 --2- 172.21.15.137:0/96253382 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ffaf80698c0 0x7ffaf81e63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 --2- 172.21.15.137:0/96253382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf806c210 0x7ffaf814c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 >> 172.21.15.137:0/96253382 conn(0x7ffaf80c5320 msgr2=0x7ffaf80c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 shutdown_connections 2022-11-27T14:34:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:56.998+0000 7ffb006a2700 1 -- 172.21.15.137:0/96253382 wait complete. 2022-11-27T14:34:57.011 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new b7d6bc6e-1be5-4355-8c30-6ae75fc573e6 3 2022-11-27T14:34:57.046 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:57.046+0000 7fb8f84b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T14:34:57.132 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.131+0000 7f6a446b0700 1 Processor -- start 2022-11-27T14:34:57.132 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.131+0000 7f6a446b0700 1 -- start start 2022-11-27T14:34:57.135 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.132+0000 7f6a446b0700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6a3c06d030 0x7f6a3c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.136 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.132+0000 7f6a446b0700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c06b120 0x7f6a3c0c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.136 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.132+0000 7f6a446b0700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c150810 con 0x7f6a3c0c0980 2022-11-27T14:34:57.136 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.132+0000 7f6a446b0700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3c0a50f0 con 0x7f6a3c06d030 2022-11-27T14:34:57.136 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.132+0000 7f6a446b0700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c150de0 con 0x7f6a3c06b120 2022-11-27T14:34:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.132+0000 7f6a41c4b700 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f6a3c0c0980 0x7f6a3c0c63e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:48580/0 (socket says 172.21.15.137:48580) 2022-11-27T14:34:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.132+0000 7f6a41c4b700 1 -- 172.21.15.137:0/2410438457 learned_addr learned my addr 172.21.15.137:0/2410438457 (peer_addr_for_me v1:172.21.15.137:0/0) 2022-11-27T14:34:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.132+0000 7f6a42c4d700 1 --2- 172.21.15.137:0/2410438457 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c06b120 0x7f6a3c0c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.137 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a4244c700 1 --2- 172.21.15.137:0/2410438457 >> v2:172.21.15.99:3300/0 conn(0x7f6a3c06d030 0x7f6a3c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.138 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a4144a700 1 -- 172.21.15.137:0/2410438457 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938386507 0 0) 0x7f6a3c150810 con 0x7f6a3c0c0980 2022-11-27T14:34:57.138 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a4144a700 1 -- 172.21.15.137:0/2410438457 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a240035b0 con 0x7f6a3c0c0980 2022-11-27T14:34:57.138 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a42c4d700 1 -- 172.21.15.137:0/2410438457 >> v2:172.21.15.99:3300/0 conn(0x7f6a3c06d030 msgr2=0x7f6a3c06fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:57.138 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a42c4d700 1 --2- 172.21.15.137:0/2410438457 >> v2:172.21.15.99:3300/0 conn(0x7f6a3c06d030 0x7f6a3c06fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a42c4d700 1 -- 172.21.15.137:0/2410438457 >> v1:172.21.15.137:6789/0 conn(0x7f6a3c0c0980 legacy=0x7f6a3c0c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a42c4d700 1 -- 172.21.15.137:0/2410438457 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c0a5420 con 0x7f6a3c06b120 2022-11-27T14:34:57.139 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a42c4d700 1 --2- 172.21.15.137:0/2410438457 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c06b120 0x7f6a3c0c4df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009d90 tx=0x7f6a3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46fdc28bdfe8ee3d server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:57.140 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a4144a700 1 -- 172.21.15.137:0/2410438457 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3400e070 con 0x7f6a3c06b120 2022-11-27T14:34:57.140 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a4144a700 1 -- 172.21.15.137:0/2410438457 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3400bb20 con 0x7f6a3c06b120 2022-11-27T14:34:57.141 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.133+0000 7f6a4144a700 1 -- 172.21.15.137:0/2410438457 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a34012430 con 0x7f6a3c06b120 2022-11-27T14:34:57.141 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.134+0000 7f6a446b0700 1 -- 172.21.15.137:0/2410438457 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c06b120 msgr2=0x7f6a3c0c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.142 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.134+0000 7f6a446b0700 1 --2- 172.21.15.137:0/2410438457 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c06b120 0x7f6a3c0c4df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009d90 tx=0x7f6a3400b330 comp rx=0 tx=0).stop 2022-11-27T14:34:57.142 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.134+0000 7f6a446b0700 1 -- 172.21.15.137:0/2410438457 shutdown_connections 2022-11-27T14:34:57.143 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.134+0000 7f6a446b0700 1 --2- 172.21.15.137:0/2410438457 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c06b120 0x7f6a3c0c4df0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.143 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.134+0000 7f6a446b0700 1 --2- 172.21.15.137:0/2410438457 >> v2:172.21.15.99:3300/0 conn(0x7f6a3c06d030 0x7f6a3c06fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.143 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.134+0000 7f6a446b0700 1 -- 172.21.15.137:0/2410438457 >> 172.21.15.137:0/2410438457 conn(0x7f6a3c0a84b0 msgr2=0x7f6a3c0c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.144 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.134+0000 7f6a446b0700 1 -- 172.21.15.137:0/2410438457 shutdown_connections 2022-11-27T14:34:57.144 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.134+0000 7f6a446b0700 1 -- 172.21.15.137:0/2410438457 wait complete. 2022-11-27T14:34:57.145 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.135+0000 7f6a446b0700 1 Processor -- start 2022-11-27T14:34:57.145 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.135+0000 7f6a446b0700 1 -- start start 2022-11-27T14:34:57.148 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a446b0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a3c06b120 0x7f6a3c0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.148 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a446b0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c06d030 0x7f6a3c0a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.148 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a446b0700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c0c0980 0x7f6a3c0a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.148 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a446b0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c14b870 con 0x7f6a3c06b120 2022-11-27T14:34:57.148 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a446b0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c14b060 con 0x7f6a3c06d030 2022-11-27T14:34:57.149 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a446b0700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1db0a0 con 0x7f6a3c0c0980 2022-11-27T14:34:57.149 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c06d030 0x7f6a3c0a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.149 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a42c4d700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c0c0980 0x7f6a3c0a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.149 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a4244c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a3c06b120 0x7f6a3c0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.150 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c06d030 0x7f6a3c0a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46514/0 (socket says 172.21.15.137:46514) 2022-11-27T14:34:57.150 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 -- 172.21.15.137:0/857172003 learned_addr learned my addr 172.21.15.137:0/857172003 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:57.150 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a42c4d700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c0c0980 0x7f6a3c0a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:42538/0 (socket says 172.21.15.137:42538) 2022-11-27T14:34:57.150 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a4244c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a3c06b120 0x7f6a3c0695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.137:46126/0 (socket says 172.21.15.137:46126) 2022-11-27T14:34:57.151 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 -- 172.21.15.137:0/857172003 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c0c0980 msgr2=0x7f6a3c0a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.151 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c0c0980 0x7f6a3c0a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.151 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 -- 172.21.15.137:0/857172003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a3c06b120 msgr2=0x7f6a3c0695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.151 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a3c06b120 0x7f6a3c0695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.152 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 -- 172.21.15.137:0/857172003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a34009a70 con 0x7f6a3c06d030 2022-11-27T14:34:57.152 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a42c4d700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c0c0980 0x7f6a3c0a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:57.152 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a4244c700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a3c06b120 0x7f6a3c0695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:57.152 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a41c4b700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c06d030 0x7f6a3c0a4190 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c700 tx=0x7f6a2c00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:57.152 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a2affd700 1 -- 172.21.15.137:0/857172003 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c005d70 con 0x7f6a3c06d030 2022-11-27T14:34:57.153 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a2affd700 1 -- 172.21.15.137:0/857172003 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0070d0 con 0x7f6a3c06d030 2022-11-27T14:34:57.153 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a2affd700 1 -- 172.21.15.137:0/857172003 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c007460 con 0x7f6a3c06d030 2022-11-27T14:34:57.153 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c069f30 con 0x7f6a3c06d030 2022-11-27T14:34:57.153 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.136+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c14b590 con 0x7f6a3c06d030 2022-11-27T14:34:57.153 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.137+0000 7f6a2affd700 1 -- 172.21.15.137:0/857172003 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f6a2c019030 con 0x7f6a3c06d030 2022-11-27T14:34:57.154 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.137+0000 7f6a2affd700 1 -- 172.21.15.137:0/857172003 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f6a2c01b840 con 0x7f6a3c06d030 2022-11-27T14:34:57.154 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.137+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c071910 con 0x7f6a3c06d030 2022-11-27T14:34:57.154 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.139+0000 7f6a2affd700 1 -- 172.21.15.137:0/857172003 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f6a2c017070 con 0x7f6a3c06d030 2022-11-27T14:34:57.179 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:57.178+0000 7fb8f84b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T14:34:57.222 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.221+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b7d6bc6e-1be5-4355-8c30-6ae75fc573e6", "id": 3} v 0) v1 -- 0x7f6a3c0c2510 con 0x7f6a3c06d030 2022-11-27T14:34:57.230 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.229+0000 7f6a2affd700 1 -- 172.21.15.137:0/857172003 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b7d6bc6e-1be5-4355-8c30-6ae75fc573e6", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6a2c01c550 con 0x7f6a3c06d030 2022-11-27T14:34:57.230 INFO:teuthology.orchestra.run.smithi137.stdout:3 2022-11-27T14:34:57.231 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.230+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c06d030 msgr2=0x7f6a3c0a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.232 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.230+0000 7f6a446b0700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c06d030 0x7f6a3c0a4190 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c700 tx=0x7f6a2c00ca10 comp rx=0 tx=0).stop 2022-11-27T14:34:57.232 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.230+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 shutdown_connections 2022-11-27T14:34:57.232 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.231+0000 7f6a446b0700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6a3c0c0980 0x7f6a3c0a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.232 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.231+0000 7f6a446b0700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a3c06b120 0x7f6a3c0695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.233 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.231+0000 7f6a446b0700 1 --2- 172.21.15.137:0/857172003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a3c06d030 0x7f6a3c0a4190 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.233 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.231+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 >> 172.21.15.137:0/857172003 conn(0x7f6a3c0a84b0 msgr2=0x7f6a3c0a6710 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.233 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.231+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 shutdown_connections 2022-11-27T14:34:57.233 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.231+0000 7f6a446b0700 1 -- 172.21.15.137:0/857172003 wait complete. 2022-11-27T14:34:57.243 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 983ec37a-caef-4592-81b1-27347e23ee68 4 2022-11-27T14:34:57.257 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:57.256+0000 7f9b369a1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T14:34:57.262 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:57.262+0000 7fb8f84b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T14:34:57.382 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.381+0000 7fd2331f0700 1 Processor -- start 2022-11-27T14:34:57.382 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.381+0000 7fd2331f0700 1 -- start start 2022-11-27T14:34:57.388 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.382+0000 7fd2331f0700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c06fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.389 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.382+0000 7fd2331f0700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c150810 con 0x7fd22c0c0980 2022-11-27T14:34:57.389 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.382+0000 7fd2331f0700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c0a5420 con 0x7fd22c06b120 2022-11-27T14:34:57.389 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.382+0000 7fd2331f0700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0a50f0 con 0x7fd22c06d030 2022-11-27T14:34:57.389 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.382+0000 7fd230f8c700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c06fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.389 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.382+0000 7fd23178d700 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7fd22c06b120 0x7fd22c0c4df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:48594/0 (socket says 172.21.15.137:48594) 2022-11-27T14:34:57.390 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.382+0000 7fd23178d700 1 -- 172.21.15.137:0/2525990613 learned_addr learned my addr 172.21.15.137:0/2525990613 (peer_addr_for_me v1:172.21.15.137:0/0) 2022-11-27T14:34:57.390 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd22b7fe700 1 -- 172.21.15.137:0/2525990613 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041863234 0 0) 0x7fd22c0a5420 con 0x7fd22c06b120 2022-11-27T14:34:57.390 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd22b7fe700 1 -- 172.21.15.137:0/2525990613 --> v1:172.21.15.137:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2100035b0 con 0x7fd22c06b120 2022-11-27T14:34:57.391 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd230f8c700 1 -- 172.21.15.137:0/2525990613 >> v1:172.21.15.137:6789/0 conn(0x7fd22c06b120 legacy=0x7fd22c0c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.391 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd230f8c700 1 -- 172.21.15.137:0/2525990613 >> v1:172.21.15.99:6789/0 conn(0x7fd22c0c0980 legacy=0x7fd22c0c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.391 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd230f8c700 1 -- 172.21.15.137:0/2525990613 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c150a70 con 0x7fd22c06d030 2022-11-27T14:34:57.392 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd230f8c700 1 --2- 172.21.15.137:0/2525990613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c06fc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd220009870 tx=0x7fd220009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de379794aec12ff server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:57.392 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd22b7fe700 1 -- 172.21.15.137:0/2525990613 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220011830 con 0x7fd22c06d030 2022-11-27T14:34:57.392 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd22b7fe700 1 -- 172.21.15.137:0/2525990613 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd220011990 con 0x7fd22c06d030 2022-11-27T14:34:57.392 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.383+0000 7fd22b7fe700 1 -- 172.21.15.137:0/2525990613 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220011d20 con 0x7fd22c06d030 2022-11-27T14:34:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.384+0000 7fd2331f0700 1 -- 172.21.15.137:0/2525990613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 msgr2=0x7fd22c06fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.384+0000 7fd2331f0700 1 --2- 172.21.15.137:0/2525990613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c06fc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd220009870 tx=0x7fd220009b80 comp rx=0 tx=0).stop 2022-11-27T14:34:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.384+0000 7fd2331f0700 1 -- 172.21.15.137:0/2525990613 shutdown_connections 2022-11-27T14:34:57.393 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.384+0000 7fd2331f0700 1 --2- 172.21.15.137:0/2525990613 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c06fc20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.394 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.384+0000 7fd2331f0700 1 -- 172.21.15.137:0/2525990613 >> 172.21.15.137:0/2525990613 conn(0x7fd22c0a84b0 msgr2=0x7fd22c0c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.394 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.384+0000 7fd2331f0700 1 -- 172.21.15.137:0/2525990613 shutdown_connections 2022-11-27T14:34:57.394 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.384+0000 7fd2331f0700 1 -- 172.21.15.137:0/2525990613 wait complete. 2022-11-27T14:34:57.395 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.385+0000 7fd2331f0700 1 Processor -- start 2022-11-27T14:34:57.395 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.385+0000 7fd2331f0700 1 -- start start 2022-11-27T14:34:57.395 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2331f0700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd22c06b120 0x7fd22c0a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.396 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2331f0700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c14b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.396 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2331f0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c0c0980 0x7fd22c14be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.396 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2331f0700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0b9860 con 0x7fd22c06b120 2022-11-27T14:34:57.396 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2331f0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c14c3b0 con 0x7fd22c0c0980 2022-11-27T14:34:57.397 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2331f0700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0a93f0 con 0x7fd22c06d030 2022-11-27T14:34:57.397 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c0c0980 0x7fd22c14be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.397 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd22bfff700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c14b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.397 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c0c0980 0x7fd22c14be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46528/0 (socket says 172.21.15.137:46528) 2022-11-27T14:34:57.398 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd230f8c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd22c06b120 0x7fd22c0a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.398 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 -- 172.21.15.137:0/1044286601 learned_addr learned my addr 172.21.15.137:0/1044286601 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:57.398 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 -- 172.21.15.137:0/1044286601 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 msgr2=0x7fd22c14b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.398 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c14b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.399 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 -- 172.21.15.137:0/1044286601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd22c06b120 msgr2=0x7fd22c0a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.399 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd22c06b120 0x7fd22c0a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.399 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 -- 172.21.15.137:0/1044286601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2200094a0 con 0x7fd22c0c0980 2022-11-27T14:34:57.399 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd230f8c700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd22c06b120 0x7fd22c0a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:57.400 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd22bfff700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c14b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:57.400 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd23178d700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c0c0980 0x7fd22c14be70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd22400cad0 tx=0x7fd22400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:57.400 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2297fa700 1 -- 172.21.15.137:0/1044286601 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22400a9a0 con 0x7fd22c0c0980 2022-11-27T14:34:57.400 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2297fa700 1 -- 172.21.15.137:0/1044286601 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22400ab00 con 0x7fd22c0c0980 2022-11-27T14:34:57.400 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c0c5640 con 0x7fd22c0c0980 2022-11-27T14:34:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.386+0000 7fd2297fa700 1 -- 172.21.15.137:0/1044286601 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22400ae90 con 0x7fd22c0c0980 2022-11-27T14:34:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.387+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c0c5c00 con 0x7fd22c0c0980 2022-11-27T14:34:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.387+0000 7fd2297fa700 1 -- 172.21.15.137:0/1044286601 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd224019030 con 0x7fd22c0c0980 2022-11-27T14:34:57.401 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.387+0000 7fd2297fa700 1 -- 172.21.15.137:0/1044286601 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fd22401b8b0 con 0x7fd22c0c0980 2022-11-27T14:34:57.402 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.387+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22c0c2550 con 0x7fd22c0c0980 2022-11-27T14:34:57.402 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.389+0000 7fd2297fa700 1 -- 172.21.15.137:0/1044286601 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fd224017070 con 0x7fd22c0c0980 2022-11-27T14:34:57.474 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.473+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "983ec37a-caef-4592-81b1-27347e23ee68", "id": 4} v 0) v1 -- 0x7fd22c0733f0 con 0x7fd22c0c0980 2022-11-27T14:34:57.477 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.477+0000 7fd2297fa700 1 -- 172.21.15.137:0/1044286601 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "983ec37a-caef-4592-81b1-27347e23ee68", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd22401bc20 con 0x7fd22c0c0980 2022-11-27T14:34:57.477 INFO:teuthology.orchestra.run.smithi137.stdout:4 2022-11-27T14:34:57.479 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c0c0980 msgr2=0x7fd22c14be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.479 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c0c0980 0x7fd22c14be70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd22400cad0 tx=0x7fd22400cde0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.479 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 shutdown_connections 2022-11-27T14:34:57.479 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd22c0c0980 0x7fd22c14be70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.480 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd22c06b120 0x7fd22c0a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.480 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 --2- 172.21.15.137:0/1044286601 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fd22c06d030 0x7fd22c14b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.480 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 >> 172.21.15.137:0/1044286601 conn(0x7fd22c0a84b0 msgr2=0x7fd22c14c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.480 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 shutdown_connections 2022-11-27T14:34:57.481 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.478+0000 7fd2331f0700 1 -- 172.21.15.137:0/1044286601 wait complete. 2022-11-27T14:34:57.490 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new a7625e4c-0f53-4600-be02-c6d0268818bd 5 2022-11-27T14:34:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.609+0000 7f510ad3b700 1 Processor -- start 2022-11-27T14:34:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.609+0000 7f510ad3b700 1 -- start start 2022-11-27T14:34:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f510ad3b700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f51040a4ed0 0x7f5104069750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f510ad3b700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51040bfa90 con 0x7f5104152a50 2022-11-27T14:34:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f510ad3b700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51040c1cb0 con 0x7f5104069380 2022-11-27T14:34:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f510ad3b700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040c2230 con 0x7f51040a4ed0 2022-11-27T14:34:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f5103fff700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f51040a4ed0 0x7f5104069750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f5108ad7700 1 --1- >> v1:172.21.15.137:6789/0 conn(0x7f5104069380 0x7f5104156a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:48602/0 (socket says 172.21.15.137:48602) 2022-11-27T14:34:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f5103fff700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f51040a4ed0 0x7f5104069750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:42554/0 (socket says 172.21.15.137:42554) 2022-11-27T14:34:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f5108ad7700 1 -- 172.21.15.137:0/3572704145 learned_addr learned my addr 172.21.15.137:0/3572704145 (peer_addr_for_me v1:172.21.15.137:0/0) 2022-11-27T14:34:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f51037fe700 1 -- 172.21.15.137:0/3572704145 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201824959 0 0) 0x7f51040c1cb0 con 0x7f5104069380 2022-11-27T14:34:57.615 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f5103fff700 1 -- 172.21.15.137:0/3572704145 >> v1:172.21.15.137:6789/0 conn(0x7f5104069380 legacy=0x7f5104156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.615 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f5103fff700 1 -- 172.21.15.137:0/3572704145 >> v1:172.21.15.99:6789/0 conn(0x7f5104152a50 legacy=0x7f5104152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.615 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f5103fff700 1 -- 172.21.15.137:0/3572704145 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51040c1f50 con 0x7f51040a4ed0 2022-11-27T14:34:57.615 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f5103fff700 1 --2- 172.21.15.137:0/3572704145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f51040a4ed0 0x7f5104069750 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009870 tx=0x7f50f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ca6a0634c6c318 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:57.615 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f51037fe700 1 -- 172.21.15.137:0/3572704145 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0004230 con 0x7f51040a4ed0 2022-11-27T14:34:57.616 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f51037fe700 1 -- 172.21.15.137:0/3572704145 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0004390 con 0x7f51040a4ed0 2022-11-27T14:34:57.616 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.610+0000 7f51037fe700 1 -- 172.21.15.137:0/3572704145 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00036a0 con 0x7f51040a4ed0 2022-11-27T14:34:57.616 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.611+0000 7f510ad3b700 1 -- 172.21.15.137:0/3572704145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f51040a4ed0 msgr2=0x7f5104069750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.616 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.611+0000 7f510ad3b700 1 --2- 172.21.15.137:0/3572704145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f51040a4ed0 0x7f5104069750 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009870 tx=0x7f50f0009b80 comp rx=0 tx=0).stop 2022-11-27T14:34:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.611+0000 7f510ad3b700 1 -- 172.21.15.137:0/3572704145 shutdown_connections 2022-11-27T14:34:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.611+0000 7f510ad3b700 1 --2- 172.21.15.137:0/3572704145 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f51040a4ed0 0x7f5104069750 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.611+0000 7f510ad3b700 1 -- 172.21.15.137:0/3572704145 >> 172.21.15.137:0/3572704145 conn(0x7f51040a7760 msgr2=0x7f51040a7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.617 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.611+0000 7f510ad3b700 1 -- 172.21.15.137:0/3572704145 shutdown_connections 2022-11-27T14:34:57.618 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.611+0000 7f510ad3b700 1 -- 172.21.15.137:0/3572704145 wait complete. 2022-11-27T14:34:57.618 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.612+0000 7f510ad3b700 1 Processor -- start 2022-11-27T14:34:57.618 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.612+0000 7f510ad3b700 1 -- start start 2022-11-27T14:34:57.621 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f510ad3b700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5104069380 0x7f51041ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.621 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f510ad3b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51040a4ed0 0x7f510406b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.621 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f510ad3b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104152a50 0x7f5104147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.622 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f510ad3b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51040a55b0 con 0x7f5104152a50 2022-11-27T14:34:57.622 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f510ad3b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51041ecb60 con 0x7f51040a4ed0 2022-11-27T14:34:57.622 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f510ad3b700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510406fbb0 con 0x7f5104069380 2022-11-27T14:34:57.622 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5103fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51040a4ed0 0x7f510406b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.622 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5108ad7700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5104069380 0x7f51041ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.623 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5103fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51040a4ed0 0x7f510406b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46538/0 (socket says 172.21.15.137:46538) 2022-11-27T14:34:57.623 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5103fff700 1 -- 172.21.15.137:0/73449284 learned_addr learned my addr 172.21.15.137:0/73449284 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:57.623 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f51092d8700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104152a50 0x7f5104147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.623 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5103fff700 1 -- 172.21.15.137:0/73449284 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5104069380 msgr2=0x7f51041ebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.624 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5103fff700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5104069380 0x7f51041ebe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.624 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5103fff700 1 -- 172.21.15.137:0/73449284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104152a50 msgr2=0x7f5104147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.624 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5103fff700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104152a50 0x7f5104147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.624 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5103fff700 1 -- 172.21.15.137:0/73449284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f00094a0 con 0x7f51040a4ed0 2022-11-27T14:34:57.625 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.613+0000 7f5108ad7700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5104069380 0x7f51041ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:57.625 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.614+0000 7f5103fff700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51040a4ed0 0x7f510406b990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f50f00090b0 tx=0x7f50f0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:57.625 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.614+0000 7f51017fa700 1 -- 172.21.15.137:0/73449284 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0011630 con 0x7f51040a4ed0 2022-11-27T14:34:57.626 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.614+0000 7f51017fa700 1 -- 172.21.15.137:0/73449284 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f001f070 con 0x7f51040a4ed0 2022-11-27T14:34:57.626 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.614+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51041ec880 con 0x7f51040a4ed0 2022-11-27T14:34:57.627 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.614+0000 7f51017fa700 1 -- 172.21.15.137:0/73449284 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00119d0 con 0x7f51040a4ed0 2022-11-27T14:34:57.627 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.614+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51040706c0 con 0x7f51040a4ed0 2022-11-27T14:34:57.627 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.615+0000 7f51017fa700 1 -- 172.21.15.137:0/73449284 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f50f0017020 con 0x7f51040a4ed0 2022-11-27T14:34:57.628 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.615+0000 7f51017fa700 1 -- 172.21.15.137:0/73449284 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f50f0021070 con 0x7f51040a4ed0 2022-11-27T14:34:57.628 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.615+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510406d210 con 0x7f51040a4ed0 2022-11-27T14:34:57.629 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.617+0000 7f51017fa700 1 -- 172.21.15.137:0/73449284 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f50f00456d0 con 0x7f51040a4ed0 2022-11-27T14:34:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.700+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7625e4c-0f53-4600-be02-c6d0268818bd", "id": 5} v 0) v1 -- 0x7f510406d3c0 con 0x7f51040a4ed0 2022-11-27T14:34:57.704 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.703+0000 7f51017fa700 1 -- 172.21.15.137:0/73449284 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7625e4c-0f53-4600-be02-c6d0268818bd", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f510406d3c0 con 0x7f51040a4ed0 2022-11-27T14:34:57.704 INFO:teuthology.orchestra.run.smithi137.stdout:5 2022-11-27T14:34:57.705 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.704+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51040a4ed0 msgr2=0x7f510406b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.706 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.704+0000 7f510ad3b700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51040a4ed0 0x7f510406b990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f50f00090b0 tx=0x7f50f0004750 comp rx=0 tx=0).stop 2022-11-27T14:34:57.706 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.704+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 shutdown_connections 2022-11-27T14:34:57.706 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.704+0000 7f510ad3b700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5104152a50 0x7f5104147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.706 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.704+0000 7f510ad3b700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5104069380 0x7f51041ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.704+0000 7f510ad3b700 1 --2- 172.21.15.137:0/73449284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51040a4ed0 0x7f510406b990 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.705+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 >> 172.21.15.137:0/73449284 conn(0x7f51040a7760 msgr2=0x7f5104152e90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.705+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 shutdown_connections 2022-11-27T14:34:57.707 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.705+0000 7f510ad3b700 1 -- 172.21.15.137:0/73449284 wait complete. 2022-11-27T14:34:57.712 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:57.712+0000 7fb8f84b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T14:34:57.717 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new c06a48ca-e3d8-45e1-b94f-9a391541ba8b 6 2022-11-27T14:34:57.815 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:57.814+0000 7f9b369a1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T14:34:57.841 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.840+0000 7ff6c8724700 1 Processor -- start 2022-11-27T14:34:57.841 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.840+0000 7ff6c8724700 1 -- start start 2022-11-27T14:34:57.844 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c8724700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff6c00c3320 0x7ff6c00c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.844 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c8724700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff6c00c50f0 0x7ff6c00c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.845 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c8724700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00c43d0 con 0x7ff6c0156d60 2022-11-27T14:34:57.845 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c8724700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00c2800 con 0x7ff6c00c50f0 2022-11-27T14:34:57.845 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c8724700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00c37c0 con 0x7ff6c00c3320 2022-11-27T14:34:57.845 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c5cbf700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff6c00c3320 0x7ff6c00c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.845 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c5cbf700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7ff6c00c3320 0x7ff6c00c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46544/0 (socket says 172.21.15.137:46544) 2022-11-27T14:34:57.846 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c5cbf700 1 -- 172.21.15.137:0/1768375560 learned_addr learned my addr 172.21.15.137:0/1768375560 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:57.846 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c64c0700 1 --1- 172.21.15.137:0/1768375560 >> v1:172.21.15.137:6789/0 conn(0x7ff6c0156d60 0x7ff6c00c6a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.137:48614/0 (socket says 172.21.15.137:48614) 2022-11-27T14:34:57.846 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.841+0000 7ff6c6cc1700 1 --2- 172.21.15.137:0/1768375560 >> v2:172.21.15.99:3300/0 conn(0x7ff6c00c50f0 0x7ff6c00c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.846 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c5cbf700 1 -- 172.21.15.137:0/1768375560 >> v2:172.21.15.99:3300/0 conn(0x7ff6c00c50f0 msgr2=0x7ff6c00c1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:57.847 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c5cbf700 1 --2- 172.21.15.137:0/1768375560 >> v2:172.21.15.99:3300/0 conn(0x7ff6c00c50f0 0x7ff6c00c1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.847 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c5cbf700 1 -- 172.21.15.137:0/1768375560 >> v1:172.21.15.137:6789/0 conn(0x7ff6c0156d60 legacy=0x7ff6c00c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.847 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c5cbf700 1 -- 172.21.15.137:0/1768375560 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00c3aa0 con 0x7ff6c00c3320 2022-11-27T14:34:57.847 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c54be700 1 -- 172.21.15.137:0/1768375560 <== mon.1 v1:172.21.15.137:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076506167 0 0) 0x7ff6c00c43d0 con 0x7ff6c0156d60 2022-11-27T14:34:57.848 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c5cbf700 1 --2- 172.21.15.137:0/1768375560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c00c3320 0x7ff6c00c1780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00c7670 tx=0x7ff6b0009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d3b998552e1980 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:57.848 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c54be700 1 -- 172.21.15.137:0/1768375560 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0011830 con 0x7ff6c00c3320 2022-11-27T14:34:57.848 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c54be700 1 -- 172.21.15.137:0/1768375560 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b0011990 con 0x7ff6c00c3320 2022-11-27T14:34:57.848 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c54be700 1 -- 172.21.15.137:0/1768375560 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0011d20 con 0x7ff6c00c3320 2022-11-27T14:34:57.848 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c8724700 1 -- 172.21.15.137:0/1768375560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c00c3320 msgr2=0x7ff6c00c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.849 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.842+0000 7ff6c8724700 1 --2- 172.21.15.137:0/1768375560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c00c3320 0x7ff6c00c1780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00c7670 tx=0x7ff6b0009bd0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.849 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.843+0000 7ff6c8724700 1 -- 172.21.15.137:0/1768375560 shutdown_connections 2022-11-27T14:34:57.849 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.843+0000 7ff6c8724700 1 --2- 172.21.15.137:0/1768375560 >> v2:172.21.15.99:3300/0 conn(0x7ff6c00c50f0 0x7ff6c00c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.849 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.843+0000 7ff6c8724700 1 --2- 172.21.15.137:0/1768375560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c00c3320 0x7ff6c00c1780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.850 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.843+0000 7ff6c8724700 1 -- 172.21.15.137:0/1768375560 >> 172.21.15.137:0/1768375560 conn(0x7ff6c006a010 msgr2=0x7ff6c00a8770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.850 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.843+0000 7ff6c8724700 1 -- 172.21.15.137:0/1768375560 shutdown_connections 2022-11-27T14:34:57.850 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.843+0000 7ff6c8724700 1 -- 172.21.15.137:0/1768375560 wait complete. 2022-11-27T14:34:57.850 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.844+0000 7ff6c8724700 1 Processor -- start 2022-11-27T14:34:57.851 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.844+0000 7ff6c8724700 1 -- start start 2022-11-27T14:34:57.853 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.844+0000 7ff6c8724700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c00c3320 0x7ff6c006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.853 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.844+0000 7ff6c8724700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff6c00c50f0 0x7ff6c00a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.853 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.844+0000 7ff6c8724700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0156d60 0x7ff6c00a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:57.854 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.844+0000 7ff6c8724700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00aa890 con 0x7ff6c00c3320 2022-11-27T14:34:57.854 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.844+0000 7ff6c8724700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00ac270 con 0x7ff6c0156d60 2022-11-27T14:34:57.854 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.844+0000 7ff6c8724700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00ac550 con 0x7ff6c00c50f0 2022-11-27T14:34:57.854 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0156d60 0x7ff6c00a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.855 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c5cbf700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff6c00c50f0 0x7ff6c00a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.855 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0156d60 0x7ff6c00a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46552/0 (socket says 172.21.15.137:46552) 2022-11-27T14:34:57.855 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 -- 172.21.15.137:0/2667154036 learned_addr learned my addr 172.21.15.137:0/2667154036 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:57.855 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c64c0700 1 --2- 172.21.15.137:0/2667154036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c00c3320 0x7ff6c006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 -- 172.21.15.137:0/2667154036 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff6c00c50f0 msgr2=0x7ff6c00a7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 --2- 172.21.15.137:0/2667154036 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff6c00c50f0 0x7ff6c00a7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 -- 172.21.15.137:0/2667154036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c00c3320 msgr2=0x7ff6c006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.856 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 --2- 172.21.15.137:0/2667154036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c00c3320 0x7ff6c006d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.857 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 -- 172.21.15.137:0/2667154036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b0009550 con 0x7ff6c0156d60 2022-11-27T14:34:57.857 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c6cc1700 1 --2- 172.21.15.137:0/2667154036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0156d60 0x7ff6c00a8460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800efc0 tx=0x7ff6b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:57.857 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6aeffd700 1 -- 172.21.15.137:0/2667154036 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8009ea0 con 0x7ff6c0156d60 2022-11-27T14:34:57.857 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6aeffd700 1 -- 172.21.15.137:0/2667154036 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b800f040 con 0x7ff6c0156d60 2022-11-27T14:34:57.857 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c00ac830 con 0x7ff6c0156d60 2022-11-27T14:34:57.858 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6aeffd700 1 -- 172.21.15.137:0/2667154036 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b800ced0 con 0x7ff6c0156d60 2022-11-27T14:34:57.858 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c00a89a0 con 0x7ff6c0156d60 2022-11-27T14:34:57.858 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6aeffd700 1 -- 172.21.15.137:0/2667154036 <== mon.1 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff6b8014480 con 0x7ff6c0156d60 2022-11-27T14:34:57.858 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.845+0000 7ff6aeffd700 1 -- 172.21.15.137:0/2667154036 <== mon.1 v2:172.21.15.137:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7ff6b80139d0 con 0x7ff6c0156d60 2022-11-27T14:34:57.859 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.846+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c00aca70 con 0x7ff6c0156d60 2022-11-27T14:34:57.859 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.848+0000 7ff6aeffd700 1 -- 172.21.15.137:0/2667154036 <== mon.1 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7ff6c00aca70 con 0x7ff6c0156d60 2022-11-27T14:34:57.932 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.931+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c06a48ca-e3d8-45e1-b94f-9a391541ba8b", "id": 6} v 0) v1 -- 0x7ff6c006acd0 con 0x7ff6c0156d60 2022-11-27T14:34:57.938 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.937+0000 7ff6aeffd700 1 -- 172.21.15.137:0/2667154036 <== mon.1 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c06a48ca-e3d8-45e1-b94f-9a391541ba8b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff6b8018020 con 0x7ff6c0156d60 2022-11-27T14:34:57.938 INFO:teuthology.orchestra.run.smithi137.stdout:6 2022-11-27T14:34:57.940 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0156d60 msgr2=0x7ff6c00a8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:57.940 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 --2- 172.21.15.137:0/2667154036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0156d60 0x7ff6c00a8460 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800efc0 tx=0x7ff6b800c3a0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 shutdown_connections 2022-11-27T14:34:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 --2- 172.21.15.137:0/2667154036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0156d60 0x7ff6c00a8460 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 --2- 172.21.15.137:0/2667154036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6c00c3320 0x7ff6c006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.941 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 --2- 172.21.15.137:0/2667154036 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff6c00c50f0 0x7ff6c00a7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 >> 172.21.15.137:0/2667154036 conn(0x7ff6c006a010 msgr2=0x7ff6c0152050 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 shutdown_connections 2022-11-27T14:34:57.942 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:57.938+0000 7ff6c8724700 1 -- 172.21.15.137:0/2667154036 wait complete. 2022-11-27T14:34:57.951 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --cluster ceph osd new 2302634c-0ed5-4467-ac3c-805b93519b9d 7 2022-11-27T14:34:57.967 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:57.966+0000 7f9b369a1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T14:34:58.074 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.073+0000 7fcb55181700 1 Processor -- start 2022-11-27T14:34:58.081 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.073+0000 7fcb55181700 1 -- start start 2022-11-27T14:34:58.087 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb55181700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb5014ce60 0x7fcb500c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.087 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb55181700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcb50152da0 0x7fcb501531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.087 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb55181700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb50150cd0 con 0x7fcb500c5480 2022-11-27T14:34:58.088 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb55181700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb50153c20 con 0x7fcb50152da0 2022-11-27T14:34:58.088 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb55181700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5014d230 con 0x7fcb5014ce60 2022-11-27T14:34:58.088 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4ed9d700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb5014ce60 0x7fcb500c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.088 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4f59e700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fcb50152da0 0x7fcb501531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.089 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4ed9d700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb5014ce60 0x7fcb500c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:42576/0 (socket says 172.21.15.137:42576) 2022-11-27T14:34:58.089 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4ed9d700 1 -- 172.21.15.137:0/3007247619 learned_addr learned my addr 172.21.15.137:0/3007247619 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:58.089 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4f59e700 1 -- 172.21.15.137:0/3007247619 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb5014ce60 msgr2=0x7fcb500c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:58.089 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4f59e700 1 --2- 172.21.15.137:0/3007247619 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb5014ce60 0x7fcb500c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.090 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4f59e700 1 -- 172.21.15.137:0/3007247619 >> v1:172.21.15.99:6789/0 conn(0x7fcb500c5480 legacy=0x7fcb5006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.090 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4f59e700 1 -- 172.21.15.137:0/3007247619 --> v2:172.21.15.137:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50071500 con 0x7fcb50152da0 2022-11-27T14:34:58.090 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4ed9d700 1 --2- 172.21.15.137:0/3007247619 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb5014ce60 0x7fcb500c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:58.090 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.074+0000 7fcb4f59e700 1 --2- 172.21.15.137:0/3007247619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50152da0 0x7fcb501531c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009d90 tx=0x7fcb4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82c8bf42881ae59 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:58.091 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb4dd9b700 1 -- 172.21.15.137:0/3007247619 <== mon.1 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4800e070 con 0x7fcb50152da0 2022-11-27T14:34:58.091 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb4dd9b700 1 -- 172.21.15.137:0/3007247619 <== mon.1 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4800bb20 con 0x7fcb50152da0 2022-11-27T14:34:58.091 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb4dd9b700 1 -- 172.21.15.137:0/3007247619 <== mon.1 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48012430 con 0x7fcb50152da0 2022-11-27T14:34:58.091 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb55181700 1 -- 172.21.15.137:0/3007247619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50152da0 msgr2=0x7fcb501531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.092 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb55181700 1 --2- 172.21.15.137:0/3007247619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50152da0 0x7fcb501531c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009d90 tx=0x7fcb4800b330 comp rx=0 tx=0).stop 2022-11-27T14:34:58.092 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb55181700 1 -- 172.21.15.137:0/3007247619 shutdown_connections 2022-11-27T14:34:58.092 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb55181700 1 --2- 172.21.15.137:0/3007247619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb50152da0 0x7fcb501531c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.092 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb55181700 1 --2- 172.21.15.137:0/3007247619 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb5014ce60 0x7fcb500c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.093 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb55181700 1 -- 172.21.15.137:0/3007247619 >> 172.21.15.137:0/3007247619 conn(0x7fcb500a77f0 msgr2=0x7fcb500a7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:58.093 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb55181700 1 -- 172.21.15.137:0/3007247619 shutdown_connections 2022-11-27T14:34:58.093 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.075+0000 7fcb55181700 1 -- 172.21.15.137:0/3007247619 wait complete. 2022-11-27T14:34:58.093 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.077+0000 7fcb55181700 1 Processor -- start 2022-11-27T14:34:58.094 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.077+0000 7fcb55181700 1 -- start start 2022-11-27T14:34:58.094 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.077+0000 7fcb55181700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb500c5480 0x7fcb50161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.094 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.077+0000 7fcb55181700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb5014ce60 0x7fcb50161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.094 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.077+0000 7fcb55181700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb50152da0 0x7fcb500a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.095 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.077+0000 7fcb55181700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb500c29a0 con 0x7fcb50152da0 2022-11-27T14:34:58.095 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.077+0000 7fcb55181700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb500c0940 con 0x7fcb5014ce60 2022-11-27T14:34:58.095 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.077+0000 7fcb55181700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb500ab360 con 0x7fcb500c5480 2022-11-27T14:34:58.095 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb500c5480 0x7fcb50161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.096 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb5014ce60 0x7fcb50161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.096 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb500c5480 0x7fcb50161a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.137:42582/0 (socket says 172.21.15.137:42582) 2022-11-27T14:34:58.096 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 -- 172.21.15.137:0/3786191876 learned_addr learned my addr 172.21.15.137:0/3786191876 (peer_addr_for_me v2:172.21.15.137:0/0) 2022-11-27T14:34:58.096 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb5014ce60 0x7fcb50161470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46562/0 (socket says 172.21.15.137:46562) 2022-11-27T14:34:58.097 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4f59e700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb50152da0 0x7fcb500a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.097 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 -- 172.21.15.137:0/3786191876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb5014ce60 msgr2=0x7fcb50161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.097 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb5014ce60 0x7fcb50161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.097 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 -- 172.21.15.137:0/3786191876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb50152da0 msgr2=0x7fcb500a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.097 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb50152da0 0x7fcb500a8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.098 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 -- 172.21.15.137:0/3786191876 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40009550 con 0x7fcb500c5480 2022-11-27T14:34:58.098 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4e59c700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb5014ce60 0x7fcb50161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:58.098 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb4ed9d700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb500c5480 0x7fcb50161a00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9e0 tx=0x7fcb4000ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:58.099 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb377fe700 1 -- 172.21.15.137:0/3786191876 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000ee80 con 0x7fcb500c5480 2022-11-27T14:34:58.099 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb377fe700 1 -- 172.21.15.137:0/3786191876 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4000c7f0 con 0x7fcb500c5480 2022-11-27T14:34:58.099 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb48009a70 con 0x7fcb500c5480 2022-11-27T14:34:58.099 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb377fe700 1 -- 172.21.15.137:0/3786191876 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000cb60 con 0x7fcb500c5480 2022-11-27T14:34:58.100 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5006da20 con 0x7fcb500c5480 2022-11-27T14:34:58.100 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb377fe700 1 -- 172.21.15.137:0/3786191876 <== mon.2 v2:172.21.15.137:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fcb40023070 con 0x7fcb500c5480 2022-11-27T14:34:58.100 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.078+0000 7fcb377fe700 1 -- 172.21.15.137:0/3786191876 <== mon.2 v2:172.21.15.137:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fcb40013b50 con 0x7fcb500c5480 2022-11-27T14:34:58.100 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.079+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb500c0c20 con 0x7fcb500c5480 2022-11-27T14:34:58.100 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.081+0000 7fcb377fe700 1 -- 172.21.15.137:0/3786191876 <== mon.2 v2:172.21.15.137:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fcb40021060 con 0x7fcb500c5480 2022-11-27T14:34:58.197 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.197+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2302634c-0ed5-4467-ac3c-805b93519b9d", "id": 7} v 0) v1 -- 0x7fcb500733f0 con 0x7fcb500c5480 2022-11-27T14:34:58.205 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.204+0000 7fcb377fe700 1 -- 172.21.15.137:0/3786191876 <== mon.2 v2:172.21.15.137:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2302634c-0ed5-4467-ac3c-805b93519b9d", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcb4002a0b0 con 0x7fcb500c5480 2022-11-27T14:34:58.205 INFO:teuthology.orchestra.run.smithi137.stdout:7 2022-11-27T14:34:58.207 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb500c5480 msgr2=0x7fcb50161a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.207 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb500c5480 0x7fcb50161a00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcb4000e9e0 tx=0x7fcb4000ecf0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.208 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 shutdown_connections 2022-11-27T14:34:58.208 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcb50152da0 0x7fcb500a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.209 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fcb500c5480 0x7fcb50161a00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.209 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 --2- 172.21.15.137:0/3786191876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb5014ce60 0x7fcb50161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.209 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 >> 172.21.15.137:0/3786191876 conn(0x7fcb500a77f0 msgr2=0x7fcb501731e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:58.209 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 shutdown_connections 2022-11-27T14:34:58.210 INFO:teuthology.orchestra.run.smithi137.stderr:2022-11-27T14:34:58.205+0000 7fcb55181700 1 -- 172.21.15.137:0/3786191876 wait complete. 2022-11-27T14:34:58.218 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-27T14:34:58.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-27T14:34:58.221 INFO:tasks.ceph.osd.0:Started 2022-11-27T14:34:58.221 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-27T14:34:58.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-27T14:34:58.224 INFO:tasks.ceph.osd.1:Started 2022-11-27T14:34:58.224 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-27T14:34:58.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-27T14:34:58.227 INFO:tasks.ceph.osd.2:Started 2022-11-27T14:34:58.227 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-27T14:34:58.227 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-27T14:34:58.230 INFO:tasks.ceph.osd.3:Started 2022-11-27T14:34:58.230 INFO:tasks.ceph.osd.4:Restarting daemon 2022-11-27T14:34:58.230 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-11-27T14:34:58.233 INFO:tasks.ceph.osd.4:Started 2022-11-27T14:34:58.233 INFO:tasks.ceph.osd.5:Restarting daemon 2022-11-27T14:34:58.233 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-11-27T14:34:58.235 INFO:tasks.ceph.osd.5:Started 2022-11-27T14:34:58.236 INFO:tasks.ceph.osd.6:Restarting daemon 2022-11-27T14:34:58.236 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-11-27T14:34:58.238 INFO:tasks.ceph.osd.6:Started 2022-11-27T14:34:58.239 INFO:tasks.ceph.osd.7:Restarting daemon 2022-11-27T14:34:58.239 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-11-27T14:34:58.242 INFO:tasks.ceph.osd.7:Started 2022-11-27T14:34:58.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T14:34:58.315 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:58.314+0000 7fb8f84b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T14:34:58.380 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.378+0000 7fe30f5a8700 1 Processor -- start 2022-11-27T14:34:58.381 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.378+0000 7fe30f5a8700 1 -- start start 2022-11-27T14:34:58.386 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30f5a8700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c6650 0x7fe3080704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.387 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30f5a8700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7fe3080c6a20 0x7fe308157120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.387 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30f5a8700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3080c1c30 0x7fe308070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.387 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30f5a8700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe308070a20 con 0x7fe3080c1c30 2022-11-27T14:34:58.388 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30f5a8700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe308070060 con 0x7fe3080c6a20 2022-11-27T14:34:58.388 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30f5a8700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080c49a0 con 0x7fe3080c6650 2022-11-27T14:34:58.388 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30db45700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3080c1c30 0x7fe308070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.388 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30db45700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3080c1c30 0x7fe308070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54690/0 (socket says 172.21.15.99:54690) 2022-11-27T14:34:58.388 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30db45700 1 -- 172.21.15.99:0/1382492409 learned_addr learned my addr 172.21.15.99:0/1382492409 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:34:58.389 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30cb43700 1 --2- 172.21.15.99:0/1382492409 >> v2:172.21.15.137:3300/0 conn(0x7fe3080c6a20 0x7fe308157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.389 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30d344700 1 --2- 172.21.15.99:0/1382492409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c6650 0x7fe3080704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.389 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.379+0000 7fe30db45700 1 -- 172.21.15.99:0/1382492409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c6650 msgr2=0x7fe3080704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:58.390 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30db45700 1 --2- 172.21.15.99:0/1382492409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c6650 0x7fe3080704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.390 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30db45700 1 -- 172.21.15.99:0/1382492409 >> v2:172.21.15.137:3300/0 conn(0x7fe3080c6a20 msgr2=0x7fe308157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:58.390 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30db45700 1 --2- 172.21.15.99:0/1382492409 >> v2:172.21.15.137:3300/0 conn(0x7fe3080c6a20 0x7fe308157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.390 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30db45700 1 -- 172.21.15.99:0/1382492409 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3080c4c80 con 0x7fe3080c1c30 2022-11-27T14:34:58.391 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30d344700 1 --2- 172.21.15.99:0/1382492409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c6650 0x7fe3080704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T14:34:58.391 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30db45700 1 --2- 172.21.15.99:0/1382492409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c1c30 0x7fe308070f00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe30806a460 tx=0x7fe30000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3d513ef64e1c5f server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:58.392 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe2f7fff700 1 -- 172.21.15.99:0/1382492409 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000e070 con 0x7fe3080c1c30 2022-11-27T14:34:58.392 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe2f7fff700 1 -- 172.21.15.99:0/1382492409 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30000b910 con 0x7fe3080c1c30 2022-11-27T14:34:58.392 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe2f7fff700 1 -- 172.21.15.99:0/1382492409 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000bcc0 con 0x7fe3080c1c30 2022-11-27T14:34:58.392 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30f5a8700 1 -- 172.21.15.99:0/1382492409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c1c30 msgr2=0x7fe308070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.393 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30f5a8700 1 --2- 172.21.15.99:0/1382492409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c1c30 0x7fe308070f00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe30806a460 tx=0x7fe30000b330 comp rx=0 tx=0).stop 2022-11-27T14:34:58.393 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30f5a8700 1 -- 172.21.15.99:0/1382492409 shutdown_connections 2022-11-27T14:34:58.393 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30f5a8700 1 --2- 172.21.15.99:0/1382492409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c1c30 0x7fe308070f00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.393 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30f5a8700 1 --2- 172.21.15.99:0/1382492409 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c6650 0x7fe3080704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.394 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30f5a8700 1 --2- 172.21.15.99:0/1382492409 >> v2:172.21.15.137:3300/0 conn(0x7fe3080c6a20 0x7fe308157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.394 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.380+0000 7fe30f5a8700 1 -- 172.21.15.99:0/1382492409 >> 172.21.15.99:0/1382492409 conn(0x7fe3080a84b0 msgr2=0x7fe3080c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:58.394 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.381+0000 7fe30f5a8700 1 -- 172.21.15.99:0/1382492409 shutdown_connections 2022-11-27T14:34:58.394 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.381+0000 7fe30f5a8700 1 -- 172.21.15.99:0/1382492409 wait complete. 2022-11-27T14:34:58.395 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30f5a8700 1 Processor -- start 2022-11-27T14:34:58.395 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30f5a8700 1 -- start start 2022-11-27T14:34:58.395 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30f5a8700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c1c30 0x7fe308152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.395 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30f5a8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c6650 0x7fe308151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.396 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30f5a8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3080aa440 0x7fe308151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:58.396 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30f5a8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080bd460 con 0x7fe3080c6650 2022-11-27T14:34:58.396 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30f5a8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080ab5d0 con 0x7fe3080aa440 2022-11-27T14:34:58.396 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30f5a8700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080ab820 con 0x7fe3080c1c30 2022-11-27T14:34:58.397 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30cb43700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c6650 0x7fe308151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.397 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30cb43700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c6650 0x7fe308151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54694/0 (socket says 172.21.15.99:54694) 2022-11-27T14:34:58.397 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.382+0000 7fe30cb43700 1 -- 172.21.15.99:0/3341824169 learned_addr learned my addr 172.21.15.99:0/3341824169 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:34:58.397 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30d344700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c1c30 0x7fe308152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.398 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30db45700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3080aa440 0x7fe308151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:58.398 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30cb43700 1 -- 172.21.15.99:0/3341824169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c1c30 msgr2=0x7fe308152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.398 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30cb43700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c1c30 0x7fe308152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.398 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30cb43700 1 -- 172.21.15.99:0/3341824169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3080aa440 msgr2=0x7fe308151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.398 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30cb43700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3080aa440 0x7fe308151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.399 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30cb43700 1 -- 172.21.15.99:0/3341824169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe300009b30 con 0x7fe3080c6650 2022-11-27T14:34:58.399 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30cb43700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c6650 0x7fe308151a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800e9b0 tx=0x7fe2f800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:58.399 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe2f5ffb700 1 -- 172.21.15.99:0/3341824169 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8009710 con 0x7fe3080c6650 2022-11-27T14:34:58.399 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe2f5ffb700 1 -- 172.21.15.99:0/3341824169 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f800ee50 con 0x7fe3080c6650 2022-11-27T14:34:58.400 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3080abb00 con 0x7fe3080c6650 2022-11-27T14:34:58.400 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe2f5ffb700 1 -- 172.21.15.99:0/3341824169 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f800c9f0 con 0x7fe3080c6650 2022-11-27T14:34:58.400 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3080bd080 con 0x7fe3080c6650 2022-11-27T14:34:58.400 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe2f5ffb700 1 -- 172.21.15.99:0/3341824169 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe2f8019030 con 0x7fe3080c6650 2022-11-27T14:34:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.383+0000 7fe2f5ffb700 1 -- 172.21.15.99:0/3341824169 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fe2f8006830 con 0x7fe3080c6650 2022-11-27T14:34:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.384+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3080733f0 con 0x7fe3080c6650 2022-11-27T14:34:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.386+0000 7fe2f5ffb700 1 -- 172.21.15.99:0/3341824169 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fe2f8021060 con 0x7fe3080c6650 2022-11-27T14:34:58.456 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.454+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3080702c0 con 0x7fe3080c6650 2022-11-27T14:34:58.457 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.455+0000 7fe2f5ffb700 1 -- 172.21.15.99:0/3341824169 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7fe2f8017070 con 0x7fe3080c6650 2022-11-27T14:34:58.457 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:34:58.458 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"fsid":"1f55be23-348c-4a1c-92de-37e343596508","created":"2022-11-27T14:34:55.727085+0000","modified":"2022-11-27T14:34:58.198186+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T14:34:58.198186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6cb243f2-dcee-4f20-87da-02657be3dce8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6206b34c-8c3d-4337-9f81-27efd1811bd4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b7d6bc6e-1be5-4355-8c30-6ae75fc573e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"983ec37a-caef-4592-81b1-27347e23ee68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7625e4c-0f53-4600-be02-c6d0268818bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c06a48ca-e3d8-45e1-b94f-9a391541ba8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2302634c-0ed5-4467-ac3c-805b93519b9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T14:34:58.459 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c6650 msgr2=0x7fe308151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:58.459 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c6650 0x7fe308151a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800e9b0 tx=0x7fe2f800ecc0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.460 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 shutdown_connections 2022-11-27T14:34:58.460 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3080aa440 0x7fe308151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.460 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fe3080c1c30 0x7fe308152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.460 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 --2- 172.21.15.99:0/3341824169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080c6650 0x7fe308151a20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:58.460 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 >> 172.21.15.99:0/3341824169 conn(0x7fe3080a84b0 msgr2=0x7fe3080c5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:58.461 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 shutdown_connections 2022-11-27T14:34:58.461 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:34:58.456+0000 7fe30f5a8700 1 -- 172.21.15.99:0/3341824169 wait complete. 2022-11-27T14:34:58.469 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-27T14:34:58.470 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-27T14:34:58.477 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:58.477+0000 7fb8f84b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T14:34:58.694 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:58.693+0000 7f9b369a1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T14:34:58.770 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:58.768+0000 7f9b369a1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T14:34:58.834 INFO:tasks.ceph.osd.0.smithi099.stderr:2022-11-27T14:34:58.832+0000 7f0fb902b5c0 -1 Falling back to public interface 2022-11-27T14:34:58.847 INFO:tasks.ceph.osd.5.smithi137.stderr:2022-11-27T14:34:58.846+0000 7fbb415d45c0 -1 Falling back to public interface 2022-11-27T14:34:58.847 INFO:tasks.ceph.osd.1.smithi099.stderr:2022-11-27T14:34:58.846+0000 7f27bf8eb5c0 -1 Falling back to public interface 2022-11-27T14:34:58.850 INFO:tasks.ceph.osd.6.smithi137.stderr:2022-11-27T14:34:58.850+0000 7f2cf7c045c0 -1 Falling back to public interface 2022-11-27T14:34:58.851 INFO:tasks.ceph.osd.2.smithi099.stderr:2022-11-27T14:34:58.850+0000 7ff5ed8a65c0 -1 Falling back to public interface 2022-11-27T14:34:58.855 INFO:tasks.ceph.osd.3.smithi099.stderr:2022-11-27T14:34:58.854+0000 7f2452f885c0 -1 Falling back to public interface 2022-11-27T14:34:58.871 INFO:tasks.ceph.osd.4.smithi137.stderr:2022-11-27T14:34:58.870+0000 7f9c82b3f5c0 -1 Falling back to public interface 2022-11-27T14:34:58.871 INFO:tasks.ceph.osd.7.smithi137.stderr:2022-11-27T14:34:58.870+0000 7fbab49c25c0 -1 Falling back to public interface 2022-11-27T14:34:58.871 INFO:tasks.ceph.osd.7.smithi137.stderr:2022-11-27T14:34:58.871+0000 7fbaabd80700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2022-11-27T14:34:58.909 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:58.907+0000 7f9b369a1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T14:34:59.064 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:59.062+0000 7f9b369a1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T14:34:59.156 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:59.155+0000 7fb8f84b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T14:34:59.195 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:59.194+0000 7f9b369a1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T14:34:59.233 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:59.232+0000 7fb8f84b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T14:34:59.275 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T14:34:59.369 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:59.367+0000 7fb8f84b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T14:34:59.391 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.389+0000 7f111ecdf700 1 Processor -- start 2022-11-27T14:34:59.391 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.389+0000 7f111ecdf700 1 -- start start 2022-11-27T14:34:59.394 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f111ecdf700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c7140 0x7f11180c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:59.395 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f111ecdf700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f11180c1a80 0x7f11180c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:59.395 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f111ecdf700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111806e090 con 0x7f11180c5360 2022-11-27T14:34:59.395 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f111ecdf700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111806d310 con 0x7f11180c1a80 2022-11-27T14:34:59.395 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f111ecdf700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180a60f0 con 0x7f11180c7140 2022-11-27T14:34:59.396 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f111ca7b700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f11180c5360 0x7f11180c5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50672/0 (socket says 172.21.15.99:50672) 2022-11-27T14:34:59.396 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f111ca7b700 1 -- 172.21.15.99:0/2655544827 learned_addr learned my addr 172.21.15.99:0/2655544827 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:34:59.396 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f1117fff700 1 --2- 172.21.15.99:0/2655544827 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c7140 0x7f11180c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:59.396 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f111d27c700 1 --2- 172.21.15.99:0/2655544827 >> v2:172.21.15.137:3300/0 conn(0x7f11180c1a80 0x7f11180c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:59.396 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f11177fe700 1 -- 172.21.15.99:0/2655544827 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670644793 0 0) 0x7f111806e090 con 0x7f11180c5360 2022-11-27T14:34:59.397 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.390+0000 7f11177fe700 1 -- 172.21.15.99:0/2655544827 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10fc0035b0 con 0x7f11180c5360 2022-11-27T14:34:59.397 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f1117fff700 1 -- 172.21.15.99:0/2655544827 >> v2:172.21.15.137:3300/0 conn(0x7f11180c1a80 msgr2=0x7f11180c3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:34:59.397 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f1117fff700 1 --2- 172.21.15.99:0/2655544827 >> v2:172.21.15.137:3300/0 conn(0x7f11180c1a80 0x7f11180c3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.397 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f1117fff700 1 -- 172.21.15.99:0/2655544827 >> v1:172.21.15.99:6789/0 conn(0x7f11180c5360 legacy=0x7f11180c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:59.398 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f1117fff700 1 -- 172.21.15.99:0/2655544827 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111806d5b0 con 0x7f11180c7140 2022-11-27T14:34:59.398 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f111d27c700 1 --2- 172.21.15.99:0/2655544827 >> v2:172.21.15.137:3300/0 conn(0x7f11180c1a80 0x7f11180c3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:34:59.398 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f1117fff700 1 --2- 172.21.15.99:0/2655544827 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c7140 0x7f11180c1540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1104009840 tx=0x7f1104009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c851f3e5db56ad server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:59.398 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f11177fe700 1 -- 172.21.15.99:0/2655544827 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104011830 con 0x7f11180c7140 2022-11-27T14:34:59.398 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f11177fe700 1 -- 172.21.15.99:0/2655544827 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1104011990 con 0x7f11180c7140 2022-11-27T14:34:59.399 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f11177fe700 1 -- 172.21.15.99:0/2655544827 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104011d20 con 0x7f11180c7140 2022-11-27T14:34:59.399 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f111ecdf700 1 -- 172.21.15.99:0/2655544827 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c7140 msgr2=0x7f11180c1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:59.399 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.391+0000 7f111ecdf700 1 --2- 172.21.15.99:0/2655544827 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c7140 0x7f11180c1540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1104009840 tx=0x7f1104009b50 comp rx=0 tx=0).stop 2022-11-27T14:34:59.399 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.392+0000 7f111ecdf700 1 -- 172.21.15.99:0/2655544827 shutdown_connections 2022-11-27T14:34:59.400 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.392+0000 7f111ecdf700 1 --2- 172.21.15.99:0/2655544827 >> v2:172.21.15.137:3300/0 conn(0x7f11180c1a80 0x7f11180c3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.400 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.392+0000 7f111ecdf700 1 --2- 172.21.15.99:0/2655544827 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c7140 0x7f11180c1540 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.400 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.392+0000 7f111ecdf700 1 -- 172.21.15.99:0/2655544827 >> 172.21.15.99:0/2655544827 conn(0x7f111806a010 msgr2=0x7f11180a8770 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:59.400 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.392+0000 7f111ecdf700 1 -- 172.21.15.99:0/2655544827 shutdown_connections 2022-11-27T14:34:59.400 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.392+0000 7f111ecdf700 1 -- 172.21.15.99:0/2655544827 wait complete. 2022-11-27T14:34:59.401 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.393+0000 7f111ecdf700 1 Processor -- start 2022-11-27T14:34:59.401 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.393+0000 7f111ecdf700 1 -- start start 2022-11-27T14:34:59.403 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111ecdf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11180c1a80 0x7f11180bc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:59.404 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111ecdf700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c5360 0x7f11180c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:59.404 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111ecdf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11180c7140 0x7f11181eb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:34:59.404 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111ecdf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111806c0b0 con 0x7f11180c7140 2022-11-27T14:34:59.404 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111ecdf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11181ebc50 con 0x7f11180c1a80 2022-11-27T14:34:59.405 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111ecdf700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180c03d0 con 0x7f11180c5360 2022-11-27T14:34:59.405 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11180c7140 0x7f11181eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:59.405 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11180c7140 0x7f11181eb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54708/0 (socket says 172.21.15.99:54708) 2022-11-27T14:34:59.405 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 -- 172.21.15.99:0/734166273 learned_addr learned my addr 172.21.15.99:0/734166273 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:34:59.406 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111ca7b700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11180c1a80 0x7f11180bc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:59.406 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f1117fff700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c5360 0x7f11180c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:34:59.406 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 -- 172.21.15.99:0/734166273 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c5360 msgr2=0x7f11180c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:59.406 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c5360 0x7f11180c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.406 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 -- 172.21.15.99:0/734166273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11180c1a80 msgr2=0x7f11180bc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:59.407 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11180c1a80 0x7f11180bc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.407 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 -- 172.21.15.99:0/734166273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110009a70 con 0x7f11180c7140 2022-11-27T14:34:59.407 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f111d27c700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11180c7140 0x7f11181eb380 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f111000efc0 tx=0x7f111000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:34:59.407 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.394+0000 7f11157fa700 1 -- 172.21.15.99:0/734166273 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000ca90 con 0x7f11180c7140 2022-11-27T14:34:59.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.395+0000 7f11157fa700 1 -- 172.21.15.99:0/734166273 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111000f040 con 0x7f11180c7140 2022-11-27T14:34:59.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.395+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11040094a0 con 0x7f11180c7140 2022-11-27T14:34:59.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.395+0000 7f11157fa700 1 -- 172.21.15.99:0/734166273 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000ce10 con 0x7f11180c7140 2022-11-27T14:34:59.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.395+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11181ebef0 con 0x7f11180c7140 2022-11-27T14:34:59.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.395+0000 7f11157fa700 1 -- 172.21.15.99:0/734166273 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f1110003a20 con 0x7f11180c7140 2022-11-27T14:34:59.409 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.395+0000 7f11157fa700 1 -- 172.21.15.99:0/734166273 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f11100047a0 con 0x7f11180c7140 2022-11-27T14:34:59.409 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.395+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111806cc60 con 0x7f11180c7140 2022-11-27T14:34:59.409 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.398+0000 7f11157fa700 1 -- 172.21.15.99:0/734166273 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f1110018030 con 0x7f11180c7140 2022-11-27T14:34:59.468 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.467+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11180733f0 con 0x7f11180c7140 2022-11-27T14:34:59.469 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.467+0000 7f11157fa700 1 -- 172.21.15.99:0/734166273 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f1110004020 con 0x7f11180c7140 2022-11-27T14:34:59.470 INFO:teuthology.misc.health.smithi099.stdout: 2022-11-27T14:34:59.470 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":10,"fsid":"1f55be23-348c-4a1c-92de-37e343596508","created":"2022-11-27T14:34:55.727085+0000","modified":"2022-11-27T14:34:58.198186+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T14:34:58.198186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6cb243f2-dcee-4f20-87da-02657be3dce8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6206b34c-8c3d-4337-9f81-27efd1811bd4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b7d6bc6e-1be5-4355-8c30-6ae75fc573e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"983ec37a-caef-4592-81b1-27347e23ee68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7625e4c-0f53-4600-be02-c6d0268818bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c06a48ca-e3d8-45e1-b94f-9a391541ba8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2302634c-0ed5-4467-ac3c-805b93519b9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T14:34:59.471 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11180c7140 msgr2=0x7f11181eb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:34:59.471 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11180c7140 0x7f11181eb380 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f111000efc0 tx=0x7f111000c3a0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.471 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 shutdown_connections 2022-11-27T14:34:59.472 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11180c7140 0x7f11181eb380 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.472 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11180c1a80 0x7f11180bc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.472 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 --2- 172.21.15.99:0/734166273 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f11180c5360 0x7f11180c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:34:59.472 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 >> 172.21.15.99:0/734166273 conn(0x7f111806a010 msgr2=0x7f11181530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:34:59.473 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 shutdown_connections 2022-11-27T14:34:59.473 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:34:59.468+0000 7f111ecdf700 1 -- 172.21.15.99:0/734166273 wait complete. 2022-11-27T14:34:59.480 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T14:34:59.527 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:59.526+0000 7fb8f84b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T14:34:59.552 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:59.551+0000 7f9b369a1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T14:34:59.656 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:34:59.655+0000 7fb8f84b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T14:34:59.703 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:59.701+0000 7f9b369a1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T14:34:59.816 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:59.814+0000 7f9b369a1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T14:34:59.959 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:34:59.958+0000 7f9b369a1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T14:35:00.012 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:35:00.010+0000 7fb8f84b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T14:35:00.056 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:35:00.055+0000 7f9b369a1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T14:35:00.162 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:35:00.160+0000 7fb8f84b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T14:35:00.278 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:35:00.277+0000 7fb8f84b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T14:35:00.403 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:35:00.401+0000 7f9b369a1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T14:35:00.423 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:35:00.422+0000 7fb8f84b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T14:35:00.522 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:35:00.521+0000 7fb8f84b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T14:35:00.882 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:35:00.881+0000 7fb8f84b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T14:35:00.973 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:35:00.972+0000 7f9b369a1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T14:35:01.470 INFO:tasks.ceph.mgr.x.smithi137.stderr:2022-11-27T14:35:01.469+0000 7fb8f84b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T14:35:01.997 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:35:01.995+0000 7f9aeed6b700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-11-27T14:35:01.997 INFO:tasks.ceph.mgr.y.smithi099.stderr:2022-11-27T14:35:01.995+0000 7f9aeed6b700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-11-27T14:35:02.362 INFO:tasks.ceph.osd.6.smithi137.stderr:2022-11-27T14:35:02.360+0000 7f2cf7c045c0 -1 osd.6 0 log_to_monitors true 2022-11-27T14:35:02.385 INFO:tasks.ceph.osd.1.smithi099.stderr:2022-11-27T14:35:02.383+0000 7f27bf8eb5c0 -1 osd.1 0 log_to_monitors true 2022-11-27T14:35:02.390 INFO:tasks.ceph.osd.2.smithi099.stderr:2022-11-27T14:35:02.388+0000 7ff5ed8a65c0 -1 osd.2 0 log_to_monitors true 2022-11-27T14:35:02.400 INFO:tasks.ceph.osd.3.smithi099.stderr:2022-11-27T14:35:02.398+0000 7f2452f885c0 -1 osd.3 0 log_to_monitors true 2022-11-27T14:35:02.401 INFO:tasks.ceph.osd.0.smithi099.stderr:2022-11-27T14:35:02.400+0000 7f0fb902b5c0 -1 osd.0 0 log_to_monitors true 2022-11-27T14:35:02.403 INFO:tasks.ceph.osd.4.smithi137.stderr:2022-11-27T14:35:02.402+0000 7f9c82b3f5c0 -1 osd.4 0 log_to_monitors true 2022-11-27T14:35:02.411 INFO:tasks.ceph.osd.5.smithi137.stderr:2022-11-27T14:35:02.410+0000 7fbb415d45c0 -1 osd.5 0 log_to_monitors true 2022-11-27T14:35:02.453 INFO:tasks.ceph.osd.7.smithi137.stderr:2022-11-27T14:35:02.452+0000 7fbab49c25c0 -1 osd.7 0 log_to_monitors true 2022-11-27T14:35:05.926 INFO:tasks.ceph.osd.6.smithi137.stderr:2022-11-27T14:35:05.922+0000 7f2ceb7bb700 -1 osd.6 0 waiting for initial osdmap 2022-11-27T14:35:05.945 INFO:tasks.ceph.osd.6.smithi137.stderr:2022-11-27T14:35:05.944+0000 7f2ce3b96700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T14:35:05.965 INFO:tasks.ceph.osd.0.smithi099.stderr:2022-11-27T14:35:05.964+0000 7f0facbe2700 -1 osd.0 0 waiting for initial osdmap 2022-11-27T14:35:05.978 INFO:tasks.ceph.osd.5.smithi137.stderr:2022-11-27T14:35:05.977+0000 7fbb3518b700 -1 osd.5 0 waiting for initial osdmap 2022-11-27T14:35:05.984 INFO:tasks.ceph.osd.0.smithi099.stderr:2022-11-27T14:35:05.983+0000 7f0fa4fbd700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T14:35:05.997 INFO:tasks.ceph.osd.4.smithi137.stderr:2022-11-27T14:35:05.995+0000 7f9c766f6700 -1 osd.4 0 waiting for initial osdmap 2022-11-27T14:35:06.000 INFO:tasks.ceph.osd.5.smithi137.stderr:2022-11-27T14:35:05.999+0000 7fbb2d566700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T14:35:06.007 INFO:tasks.ceph.osd.4.smithi137.stderr:2022-11-27T14:35:06.006+0000 7f9c6ead1700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T14:35:06.018 INFO:tasks.ceph.osd.7.smithi137.stderr:2022-11-27T14:35:06.016+0000 7fbaa8579700 -1 osd.7 0 waiting for initial osdmap 2022-11-27T14:35:06.026 INFO:tasks.ceph.osd.1.smithi099.stderr:2022-11-27T14:35:06.024+0000 7f27b34a2700 -1 osd.1 0 waiting for initial osdmap 2022-11-27T14:35:06.029 INFO:tasks.ceph.osd.3.smithi099.stderr:2022-11-27T14:35:06.027+0000 7f2446b3f700 -1 osd.3 0 waiting for initial osdmap 2022-11-27T14:35:06.029 INFO:tasks.ceph.osd.7.smithi137.stderr:2022-11-27T14:35:06.028+0000 7fbaa0954700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T14:35:06.038 INFO:tasks.ceph.osd.2.smithi099.stderr:2022-11-27T14:35:06.036+0000 7ff5e145d700 -1 osd.2 0 waiting for initial osdmap 2022-11-27T14:35:06.038 INFO:tasks.ceph.osd.1.smithi099.stderr:2022-11-27T14:35:06.037+0000 7f27ab87d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T14:35:06.052 INFO:tasks.ceph.osd.3.smithi099.stderr:2022-11-27T14:35:06.051+0000 7f243ef1a700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T14:35:06.055 INFO:tasks.ceph.osd.2.smithi099.stderr:2022-11-27T14:35:06.054+0000 7ff5d9838700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T14:35:06.288 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T14:35:06.401 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.399+0000 7fde18032700 1 Processor -- start 2022-11-27T14:35:06.401 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.399+0000 7fde18032700 1 -- start start 2022-11-27T14:35:06.407 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde18032700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fde1014d0c0 0x7fde100714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:06.407 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde18032700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:06.407 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde18032700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde100a4e50 con 0x7fde1006fa80 2022-11-27T14:35:06.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde18032700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde10071a80 con 0x7fde1014d0c0 2022-11-27T14:35:06.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde18032700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde101504b0 con 0x7fde100c3a60 2022-11-27T14:35:06.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde15dce700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fde1014d0c0 0x7fde100714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:06.408 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde15dce700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fde1014d0c0 0x7fde100714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54808/0 (socket says 172.21.15.99:54808) 2022-11-27T14:35:06.409 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde15dce700 1 -- 172.21.15.99:0/776572923 learned_addr learned my addr 172.21.15.99:0/776572923 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:06.409 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde165cf700 1 --2- 172.21.15.99:0/776572923 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:06.409 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde15dce700 1 -- 172.21.15.99:0/776572923 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 msgr2=0x7fde100c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:06.409 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde15dce700 1 --2- 172.21.15.99:0/776572923 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.410 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde15dce700 1 -- 172.21.15.99:0/776572923 >> v1:172.21.15.137:6789/0 conn(0x7fde1006fa80 legacy=0x7fde100c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:06.410 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde15dce700 1 -- 172.21.15.99:0/776572923 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100a50f0 con 0x7fde1014d0c0 2022-11-27T14:35:06.410 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.400+0000 7fde15dce700 1 --2- 172.21.15.99:0/776572923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1014d0c0 0x7fde100714a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde000098f0 tx=0x7fde00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444962904aa77a44 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:06.410 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde14dcc700 1 -- 172.21.15.99:0/776572923 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00004230 con 0x7fde1014d0c0 2022-11-27T14:35:06.410 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde14dcc700 1 -- 172.21.15.99:0/776572923 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde00004390 con 0x7fde1014d0c0 2022-11-27T14:35:06.411 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde14dcc700 1 -- 172.21.15.99:0/776572923 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000036a0 con 0x7fde1014d0c0 2022-11-27T14:35:06.411 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde18032700 1 -- 172.21.15.99:0/776572923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1014d0c0 msgr2=0x7fde100714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:06.411 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde18032700 1 --2- 172.21.15.99:0/776572923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1014d0c0 0x7fde100714a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde000098f0 tx=0x7fde00009c00 comp rx=0 tx=0).stop 2022-11-27T14:35:06.411 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde18032700 1 -- 172.21.15.99:0/776572923 shutdown_connections 2022-11-27T14:35:06.412 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde18032700 1 --2- 172.21.15.99:0/776572923 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.412 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde18032700 1 --2- 172.21.15.99:0/776572923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1014d0c0 0x7fde100714a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.412 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde18032700 1 -- 172.21.15.99:0/776572923 >> 172.21.15.99:0/776572923 conn(0x7fde100a7760 msgr2=0x7fde100a7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:06.412 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde18032700 1 -- 172.21.15.99:0/776572923 shutdown_connections 2022-11-27T14:35:06.413 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.401+0000 7fde18032700 1 -- 172.21.15.99:0/776572923 wait complete. 2022-11-27T14:35:06.413 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.402+0000 7fde18032700 1 Processor -- start 2022-11-27T14:35:06.413 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.402+0000 7fde18032700 1 -- start start 2022-11-27T14:35:06.413 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde18032700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1006fa80 0x7fde101e5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:06.413 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde18032700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:06.414 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde18032700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde1014d0c0 0x7fde10153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:06.414 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde18032700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde101e5990 con 0x7fde1006fa80 2022-11-27T14:35:06.414 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde18032700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100a9dd0 con 0x7fde1014d0c0 2022-11-27T14:35:06.414 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde18032700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1014fd00 con 0x7fde100c3a60 2022-11-27T14:35:06.415 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde15dce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1006fa80 0x7fde101e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:06.415 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde15dce700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1006fa80 0x7fde101e5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54814/0 (socket says 172.21.15.99:54814) 2022-11-27T14:35:06.415 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde15dce700 1 -- 172.21.15.99:0/2438039276 learned_addr learned my addr 172.21.15.99:0/2438039276 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:06.415 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde155cd700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:06.415 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde165cf700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde1014d0c0 0x7fde10153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:06.416 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde15dce700 1 -- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 msgr2=0x7fde100ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:06.416 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde15dce700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.416 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde15dce700 1 -- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde1014d0c0 msgr2=0x7fde10153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:06.416 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde15dce700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde1014d0c0 0x7fde10153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.417 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde15dce700 1 -- 172.21.15.99:0/2438039276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde00009550 con 0x7fde1006fa80 2022-11-27T14:35:06.417 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.403+0000 7fde155cd700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T14:35:06.417 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.404+0000 7fde15dce700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1006fa80 0x7fde101e5450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fde100a84a0 tx=0x7fde00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:06.418 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.404+0000 7fddfe7fc700 1 -- 172.21.15.99:0/2438039276 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00003e30 con 0x7fde1006fa80 2022-11-27T14:35:06.418 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.404+0000 7fddfe7fc700 1 -- 172.21.15.99:0/2438039276 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0001f070 con 0x7fde1006fa80 2022-11-27T14:35:06.419 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.404+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde100abaa0 con 0x7fde1006fa80 2022-11-27T14:35:06.419 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.404+0000 7fddfe7fc700 1 -- 172.21.15.99:0/2438039276 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0000b8a0 con 0x7fde1006fa80 2022-11-27T14:35:06.420 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.404+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde100a8cb0 con 0x7fde1006fa80 2022-11-27T14:35:06.420 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.405+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde100733f0 con 0x7fde1006fa80 2022-11-27T14:35:06.420 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.405+0000 7fddfe7fc700 1 -- 172.21.15.99:0/2438039276 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7fde0002a020 con 0x7fde1006fa80 2022-11-27T14:35:06.420 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.405+0000 7fddfe7fc700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fddf0066540 0x7fddf00689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:06.420 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.405+0000 7fddfe7fc700 1 -- 172.21.15.99:0/2438039276 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fde000840d0 con 0x7fde1006fa80 2022-11-27T14:35:06.421 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.405+0000 7fde155cd700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fddf0066540 0x7fddf00689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:06.421 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.406+0000 7fde155cd700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fddf0066540 0x7fddf00689f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde100a73c0 tx=0x7fde0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:06.422 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.410+0000 7fddfe7fc700 1 -- 172.21.15.99:0/2438039276 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fde00052720 con 0x7fde1006fa80 2022-11-27T14:35:06.592 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.591+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde100733f0 con 0x7fde1006fa80 2022-11-27T14:35:06.593 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.591+0000 7fddfe7fc700 1 -- 172.21.15.99:0/2438039276 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7011 (secure 0 0 0) 0x7fde00055d40 con 0x7fde1006fa80 2022-11-27T14:35:06.593 INFO:teuthology.misc.health.smithi099.stdout: 2022-11-27T14:35:06.594 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":12,"fsid":"1f55be23-348c-4a1c-92de-37e343596508","created":"2022-11-27T14:34:55.727085+0000","modified":"2022-11-27T14:35:03.992626+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T14:34:58.198186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6cb243f2-dcee-4f20-87da-02657be3dce8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"6206b34c-8c3d-4337-9f81-27efd1811bd4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b7d6bc6e-1be5-4355-8c30-6ae75fc573e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"983ec37a-caef-4592-81b1-27347e23ee68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7625e4c-0f53-4600-be02-c6d0268818bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c06a48ca-e3d8-45e1-b94f-9a391541ba8b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2302634c-0ed5-4467-ac3c-805b93519b9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T14:35:06.595 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fddf0066540 msgr2=0x7fddf00689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:06.596 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fddf0066540 0x7fddf00689f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde100a73c0 tx=0x7fde0c008040 comp rx=0 tx=0).stop 2022-11-27T14:35:06.596 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1006fa80 msgr2=0x7fde101e5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:06.596 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1006fa80 0x7fde101e5450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fde100a84a0 tx=0x7fde00004750 comp rx=0 tx=0).stop 2022-11-27T14:35:06.596 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 shutdown_connections 2022-11-27T14:35:06.597 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fddf0066540 0x7fddf00689f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.597 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde1014d0c0 0x7fde10153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.597 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde1006fa80 0x7fde101e5450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.597 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 --2- 172.21.15.99:0/2438039276 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fde100c3a60 0x7fde100ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:06.597 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 >> 172.21.15.99:0/2438039276 conn(0x7fde100a7760 msgr2=0x7fde100c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:06.598 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 shutdown_connections 2022-11-27T14:35:06.598 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:06.593+0000 7fde18032700 1 -- 172.21.15.99:0/2438039276 wait complete. 2022-11-27T14:35:06.606 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T14:35:13.417 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T14:35:13.542 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.540+0000 7f5afc676700 1 Processor -- start 2022-11-27T14:35:13.542 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.540+0000 7f5afc676700 1 -- start start 2022-11-27T14:35:13.545 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afc676700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5af40a3d30 0x7f5af414fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.546 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afc676700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f5af40c0f70 0x7f5af40a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.546 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afc676700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af406b650 con 0x7f5af40c14a0 2022-11-27T14:35:13.546 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afc676700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af414ce80 con 0x7f5af40a3d30 2022-11-27T14:35:13.546 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afc676700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af40bf740 con 0x7f5af40c0f70 2022-11-27T14:35:13.547 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afa412700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5af40a3d30 0x7f5af414fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.547 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afa412700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5af40a3d30 0x7f5af414fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55036/0 (socket says 172.21.15.99:55036) 2022-11-27T14:35:13.547 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afa412700 1 -- 172.21.15.99:0/3335613055 learned_addr learned my addr 172.21.15.99:0/3335613055 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:13.547 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5af9c11700 1 --2- 172.21.15.99:0/3335613055 >> v2:172.21.15.137:3300/0 conn(0x7f5af40c0f70 0x7f5af40a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.548 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afa412700 1 -- 172.21.15.99:0/3335613055 >> v2:172.21.15.137:3300/0 conn(0x7f5af40c0f70 msgr2=0x7f5af40a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:13.548 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afa412700 1 --2- 172.21.15.99:0/3335613055 >> v2:172.21.15.137:3300/0 conn(0x7f5af40c0f70 0x7f5af40a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.548 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afa412700 1 -- 172.21.15.99:0/3335613055 >> v1:172.21.15.137:6789/0 conn(0x7f5af40c14a0 legacy=0x7f5af40c02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.549 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afa412700 1 -- 172.21.15.99:0/3335613055 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40bfa20 con 0x7f5af40a3d30 2022-11-27T14:35:13.549 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5afa412700 1 --2- 172.21.15.99:0/3335613055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40a3d30 0x7f5af414fc00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5af406cdf0 tx=0x7f5af000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2403c26a559318e1 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:13.549 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5af9410700 1 -- 172.21.15.99:0/3335613055 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af000dc70 con 0x7f5af40a3d30 2022-11-27T14:35:13.549 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5af9410700 1 -- 172.21.15.99:0/3335613055 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af000ddd0 con 0x7f5af40a3d30 2022-11-27T14:35:13.549 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.541+0000 7f5af9410700 1 -- 172.21.15.99:0/3335613055 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0010690 con 0x7f5af40a3d30 2022-11-27T14:35:13.550 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.542+0000 7f5afc676700 1 -- 172.21.15.99:0/3335613055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40a3d30 msgr2=0x7f5af414fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.550 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.542+0000 7f5afc676700 1 --2- 172.21.15.99:0/3335613055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40a3d30 0x7f5af414fc00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5af406cdf0 tx=0x7f5af000be70 comp rx=0 tx=0).stop 2022-11-27T14:35:13.550 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.542+0000 7f5afc676700 1 -- 172.21.15.99:0/3335613055 shutdown_connections 2022-11-27T14:35:13.551 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.542+0000 7f5afc676700 1 --2- 172.21.15.99:0/3335613055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40a3d30 0x7f5af414fc00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.551 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.542+0000 7f5afc676700 1 --2- 172.21.15.99:0/3335613055 >> v2:172.21.15.137:3300/0 conn(0x7f5af40c0f70 0x7f5af40a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.551 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.542+0000 7f5afc676700 1 -- 172.21.15.99:0/3335613055 >> 172.21.15.99:0/3335613055 conn(0x7f5af40c49a0 msgr2=0x7f5af40c4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:13.551 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.542+0000 7f5afc676700 1 -- 172.21.15.99:0/3335613055 shutdown_connections 2022-11-27T14:35:13.552 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.542+0000 7f5afc676700 1 -- 172.21.15.99:0/3335613055 wait complete. 2022-11-27T14:35:13.552 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.544+0000 7f5afc676700 1 Processor -- start 2022-11-27T14:35:13.552 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.544+0000 7f5afc676700 1 -- start start 2022-11-27T14:35:13.556 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afc676700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5af40a3d30 0x7f5af41d9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.556 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afc676700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40c0f70 0x7f5af4149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.556 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afc676700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af40c14a0 0x7f5af41e4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.556 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afc676700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af41e3dd0 con 0x7f5af40c0f70 2022-11-27T14:35:13.557 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afc676700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af41d92a0 con 0x7f5af40c14a0 2022-11-27T14:35:13.557 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afc676700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af41e4070 con 0x7f5af40a3d30 2022-11-27T14:35:13.557 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40c0f70 0x7f5af4149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.557 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40c0f70 0x7f5af4149790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55042/0 (socket says 172.21.15.99:55042) 2022-11-27T14:35:13.557 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 -- 172.21.15.99:0/3928983575 learned_addr learned my addr 172.21.15.99:0/3928983575 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:13.558 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afa412700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5af40a3d30 0x7f5af41d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.558 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afac13700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af40c14a0 0x7f5af41e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.558 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 -- 172.21.15.99:0/3928983575 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5af40a3d30 msgr2=0x7f5af41d9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.558 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5af40a3d30 0x7f5af41d9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.559 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 -- 172.21.15.99:0/3928983575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af40c14a0 msgr2=0x7f5af41e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.559 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af40c14a0 0x7f5af41e4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.559 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 -- 172.21.15.99:0/3928983575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af000b7f0 con 0x7f5af40c0f70 2022-11-27T14:35:13.559 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5af9c11700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40c0f70 0x7f5af4149790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400e9b0 tx=0x7f5ae400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:13.559 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5ae2ffd700 1 -- 172.21.15.99:0/3928983575 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4009710 con 0x7f5af40c0f70 2022-11-27T14:35:13.560 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5ae2ffd700 1 -- 172.21.15.99:0/3928983575 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae400ee50 con 0x7f5af40c0f70 2022-11-27T14:35:13.560 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af41d8a00 con 0x7f5af40c0f70 2022-11-27T14:35:13.560 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5ae2ffd700 1 -- 172.21.15.99:0/3928983575 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400c9f0 con 0x7f5af40c0f70 2022-11-27T14:35:13.560 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.545+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af41d8fc0 con 0x7f5af40c0f70 2022-11-27T14:35:13.561 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.546+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af40733f0 con 0x7f5af40c0f70 2022-11-27T14:35:13.561 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.547+0000 7f5ae2ffd700 1 -- 172.21.15.99:0/3928983575 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f5ae4019030 con 0x7f5af40c0f70 2022-11-27T14:35:13.561 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.547+0000 7f5ae2ffd700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f5ad4066590 0x7f5ad4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.561 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.547+0000 7f5afa412700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f5ad4066590 0x7f5ad4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.561 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.547+0000 7f5ae2ffd700 1 -- 172.21.15.99:0/3928983575 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5ae4083150 con 0x7f5af40c0f70 2022-11-27T14:35:13.562 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.548+0000 7f5afa412700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f5ad4066590 0x7f5ad4068a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5af000beb0 tx=0x7f5af0013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:13.562 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.552+0000 7f5ae2ffd700 1 -- 172.21.15.99:0/3928983575 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f5ae4050d10 con 0x7f5af40c0f70 2022-11-27T14:35:13.734 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.732+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5af41e41b0 con 0x7f5af40c0f70 2022-11-27T14:35:13.735 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.732+0000 7f5ae2ffd700 1 -- 172.21.15.99:0/3928983575 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12221 (secure 0 0 0) 0x7f5ae4054330 con 0x7f5af40c0f70 2022-11-27T14:35:13.736 INFO:teuthology.misc.health.smithi099.stdout: 2022-11-27T14:35:13.736 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":16,"fsid":"1f55be23-348c-4a1c-92de-37e343596508","created":"2022-11-27T14:34:55.727085+0000","modified":"2022-11-27T14:35:10.389185+0000","last_up_change":"2022-11-27T14:35:06.926207+0000","last_in_change":"2022-11-27T14:34:58.198186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T14:35:07.033933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":101281},{"type":"v1","addr":"172.21.15.99:6801","nonce":101281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":101281},{"type":"v1","addr":"172.21.15.99:6803","nonce":101281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":101281},{"type":"v1","addr":"172.21.15.99:6807","nonce":101281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":101281},{"type":"v1","addr":"172.21.15.99:6805","nonce":101281}]},"public_addr":"172.21.15.99:6801/101281","cluster_addr":"172.21.15.99:6803/101281","heartbeat_back_addr":"172.21.15.99:6807/101281","heartbeat_front_addr":"172.21.15.99:6805/101281","state":["exists","up"]},{"osd":1,"uuid":"6cb243f2-dcee-4f20-87da-02657be3dce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":101290},{"type":"v1","addr":"172.21.15.99:6809","nonce":101290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":101290},{"type":"v1","addr":"172.21.15.99:6811","nonce":101290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":101290},{"type":"v1","addr":"172.21.15.99:6815","nonce":101290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":101290},{"type":"v1","addr":"172.21.15.99:6813","nonce":101290}]},"public_addr":"172.21.15.99:6809/101290","cluster_addr":"172.21.15.99:6811/101290","heartbeat_back_addr":"172.21.15.99:6815/101290","heartbeat_front_addr":"172.21.15.99:6813/101290","state":["exists","up"]},{"osd":2,"uuid":"6206b34c-8c3d-4337-9f81-27efd1811bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":101291},{"type":"v1","addr":"172.21.15.99:6817","nonce":101291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":101291},{"type":"v1","addr":"172.21.15.99:6819","nonce":101291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":101291},{"type":"v1","addr":"172.21.15.99:6823","nonce":101291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":101291},{"type":"v1","addr":"172.21.15.99:6821","nonce":101291}]},"public_addr":"172.21.15.99:6817/101291","cluster_addr":"172.21.15.99:6819/101291","heartbeat_back_addr":"172.21.15.99:6823/101291","heartbeat_front_addr":"172.21.15.99:6821/101291","state":["exists","up"]},{"osd":3,"uuid":"b7d6bc6e-1be5-4355-8c30-6ae75fc573e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":101298},{"type":"v1","addr":"172.21.15.99:6825","nonce":101298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":101298},{"type":"v1","addr":"172.21.15.99:6827","nonce":101298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":101298},{"type":"v1","addr":"172.21.15.99:6831","nonce":101298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":101298},{"type":"v1","addr":"172.21.15.99:6829","nonce":101298}]},"public_addr":"172.21.15.99:6825/101298","cluster_addr":"172.21.15.99:6827/101298","heartbeat_back_addr":"172.21.15.99:6831/101298","heartbeat_front_addr":"172.21.15.99:6829/101298","state":["exists","up"]},{"osd":4,"uuid":"983ec37a-caef-4592-81b1-27347e23ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":101499},{"type":"v1","addr":"172.21.15.137:6818","nonce":101499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":101499},{"type":"v1","addr":"172.21.15.137:6822","nonce":101499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":101499},{"type":"v1","addr":"172.21.15.137:6830","nonce":101499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":101499},{"type":"v1","addr":"172.21.15.137:6826","nonce":101499}]},"public_addr":"172.21.15.137:6818/101499","cluster_addr":"172.21.15.137:6822/101499","heartbeat_back_addr":"172.21.15.137:6830/101499","heartbeat_front_addr":"172.21.15.137:6826/101499","state":["exists","up"]},{"osd":5,"uuid":"a7625e4c-0f53-4600-be02-c6d0268818bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":101475},{"type":"v1","addr":"172.21.15.137:6801","nonce":101475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":101475},{"type":"v1","addr":"172.21.15.137:6803","nonce":101475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":101475},{"type":"v1","addr":"172.21.15.137:6807","nonce":101475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":101475},{"type":"v1","addr":"172.21.15.137:6805","nonce":101475}]},"public_addr":"172.21.15.137:6801/101475","cluster_addr":"172.21.15.137:6803/101475","heartbeat_back_addr":"172.21.15.137:6807/101475","heartbeat_front_addr":"172.21.15.137:6805/101475","state":["exists","up"]},{"osd":6,"uuid":"c06a48ca-e3d8-45e1-b94f-9a391541ba8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":101476},{"type":"v1","addr":"172.21.15.137:6809","nonce":101476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":101476},{"type":"v1","addr":"172.21.15.137:6811","nonce":101476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":101476},{"type":"v1","addr":"172.21.15.137:6815","nonce":101476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":101476},{"type":"v1","addr":"172.21.15.137:6813","nonce":101476}]},"public_addr":"172.21.15.137:6809/101476","cluster_addr":"172.21.15.137:6811/101476","heartbeat_back_addr":"172.21.15.137:6815/101476","heartbeat_front_addr":"172.21.15.137:6813/101476","state":["exists","up"]},{"osd":7,"uuid":"2302634c-0ed5-4467-ac3c-805b93519b9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6817","nonce":101506},{"type":"v1","addr":"172.21.15.137:6819","nonce":101506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6821","nonce":101506},{"type":"v1","addr":"172.21.15.137:6823","nonce":101506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6829","nonce":101506},{"type":"v1","addr":"172.21.15.137:6831","nonce":101506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":101506},{"type":"v1","addr":"172.21.15.137:6827","nonce":101506}]},"public_addr":"172.21.15.137:6819/101506","cluster_addr":"172.21.15.137:6823/101506","heartbeat_back_addr":"172.21.15.137:6831/101506","heartbeat_front_addr":"172.21.15.137:6827/101506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.371397+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.413296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.430729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.359936+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.443571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.389589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.404518+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.449713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T14:35:13.738 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f5ad4066590 msgr2=0x7f5ad4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.738 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f5ad4066590 0x7f5ad4068a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5af000beb0 tx=0x7f5af0013000 comp rx=0 tx=0).stop 2022-11-27T14:35:13.739 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40c0f70 msgr2=0x7f5af4149790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.739 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40c0f70 0x7f5af4149790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400e9b0 tx=0x7f5ae400ecc0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.739 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 shutdown_connections 2022-11-27T14:35:13.740 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f5ad4066590 0x7f5ad4068a40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.740 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af40c14a0 0x7f5af41e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.740 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f5af40a3d30 0x7f5af41d9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.740 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 --2- 172.21.15.99:0/3928983575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5af40c0f70 0x7f5af4149790 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.740 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.734+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 >> 172.21.15.99:0/3928983575 conn(0x7f5af40c49a0 msgr2=0x7f5af414bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:13.741 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.735+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 shutdown_connections 2022-11-27T14:35:13.741 INFO:teuthology.misc.health.smithi099.stderr:2022-11-27T14:35:13.735+0000 7f5afc676700 1 -- 172.21.15.99:0/3928983575 wait complete. 2022-11-27T14:35:13.747 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-11-27T14:35:13.748 INFO:tasks.ceph:Creating RBD pool 2022-11-27T14:35:13.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-27T14:35:13.867 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.866+0000 7f28f1b2f700 1 Processor -- start 2022-11-27T14:35:13.868 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.866+0000 7f28f1b2f700 1 -- start start 2022-11-27T14:35:13.874 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28f1b2f700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f28ec0c6940 0x7f28ec0c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.875 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28f1b2f700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f28ec0c4db0 0x7f28ec0c5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.875 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28f1b2f700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ec0c28f0 con 0x7f28ec156f00 2022-11-27T14:35:13.875 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28f1b2f700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec0c5d80 con 0x7f28ec0c4db0 2022-11-27T14:35:13.875 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28f1b2f700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec1572d0 con 0x7f28ec0c6940 2022-11-27T14:35:13.876 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ebfff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f28ec0c4db0 0x7f28ec0c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.876 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28eb7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f28ec156f00 0x7f28ec071480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51022/0 (socket says 172.21.15.99:51022) 2022-11-27T14:35:13.877 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28eb7fe700 1 -- 172.21.15.99:0/1824774446 learned_addr learned my addr 172.21.15.99:0/1824774446 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:35:13.877 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28eaffd700 1 --2- 172.21.15.99:0/1824774446 >> v2:172.21.15.137:3300/0 conn(0x7f28ec0c6940 0x7f28ec0c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.877 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ebfff700 1 -- 172.21.15.99:0/1824774446 >> v2:172.21.15.137:3300/0 conn(0x7f28ec0c6940 msgr2=0x7f28ec0c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:13.878 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ebfff700 1 --2- 172.21.15.99:0/1824774446 >> v2:172.21.15.137:3300/0 conn(0x7f28ec0c6940 0x7f28ec0c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.878 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ebfff700 1 -- 172.21.15.99:0/1824774446 >> v1:172.21.15.99:6789/0 conn(0x7f28ec156f00 legacy=0x7f28ec071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.879 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ebfff700 1 -- 172.21.15.99:0/1824774446 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0c7460 con 0x7f28ec0c4db0 2022-11-27T14:35:13.879 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ea7fc700 1 -- 172.21.15.99:0/1824774446 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303793551 0 0) 0x7f28ec0c28f0 con 0x7f28ec156f00 2022-11-27T14:35:13.880 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ebfff700 1 --2- 172.21.15.99:0/1824774446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec0c4db0 0x7f28ec0c5210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0c65b0 tx=0x7f28e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233e350d8521e5da server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:13.880 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ea7fc700 1 -- 172.21.15.99:0/1824774446 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400e070 con 0x7f28ec0c4db0 2022-11-27T14:35:13.880 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ea7fc700 1 -- 172.21.15.99:0/1824774446 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e400bbd0 con 0x7f28ec0c4db0 2022-11-27T14:35:13.881 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.867+0000 7f28ea7fc700 1 -- 172.21.15.99:0/1824774446 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4012430 con 0x7f28ec0c4db0 2022-11-27T14:35:13.881 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.868+0000 7f28f1b2f700 1 -- 172.21.15.99:0/1824774446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec0c4db0 msgr2=0x7f28ec0c5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.882 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.868+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/1824774446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec0c4db0 0x7f28ec0c5210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0c65b0 tx=0x7f28e400b330 comp rx=0 tx=0).stop 2022-11-27T14:35:13.882 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.868+0000 7f28f1b2f700 1 -- 172.21.15.99:0/1824774446 shutdown_connections 2022-11-27T14:35:13.883 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.868+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/1824774446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec0c4db0 0x7f28ec0c5210 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.883 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.868+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/1824774446 >> v2:172.21.15.137:3300/0 conn(0x7f28ec0c6940 0x7f28ec0c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.884 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.868+0000 7f28f1b2f700 1 -- 172.21.15.99:0/1824774446 >> 172.21.15.99:0/1824774446 conn(0x7f28ec069380 msgr2=0x7f28ec069780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:13.884 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.868+0000 7f28f1b2f700 1 -- 172.21.15.99:0/1824774446 shutdown_connections 2022-11-27T14:35:13.884 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.868+0000 7f28f1b2f700 1 -- 172.21.15.99:0/1824774446 wait complete. 2022-11-27T14:35:13.885 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.869+0000 7f28f1b2f700 1 Processor -- start 2022-11-27T14:35:13.885 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.869+0000 7f28f1b2f700 1 -- start start 2022-11-27T14:35:13.886 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28f1b2f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ec0c4db0 0x7f28ec0aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.886 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28f1b2f700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ec0c6940 0x7f28ec0abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.886 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28f1b2f700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec156f00 0x7f28ec14f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.887 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28f1b2f700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec0aae80 con 0x7f28ec156f00 2022-11-27T14:35:13.887 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28f1b2f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec1520b0 con 0x7f28ec0c4db0 2022-11-27T14:35:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28f1b2f700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec0ac7e0 con 0x7f28ec0c6940 2022-11-27T14:35:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec156f00 0x7f28ec14f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.888 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec156f00 0x7f28ec14f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55058/0 (socket says 172.21.15.99:55058) 2022-11-27T14:35:13.889 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 -- 172.21.15.99:0/3536991557 learned_addr learned my addr 172.21.15.99:0/3536991557 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:13.889 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28eaffd700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ec0c6940 0x7f28ec0abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.889 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28eb7fe700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ec0c4db0 0x7f28ec0aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.890 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 -- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ec0c6940 msgr2=0x7f28ec0abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.890 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ec0c6940 0x7f28ec0abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.890 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 -- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ec0c4db0 msgr2=0x7f28ec0aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:13.890 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ec0c4db0 0x7f28ec0aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:13.890 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 -- 172.21.15.99:0/3536991557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e4009a70 con 0x7f28ec156f00 2022-11-27T14:35:13.891 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28eb7fe700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ec0c4db0 0x7f28ec0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T14:35:13.891 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28ebfff700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec156f00 0x7f28ec14f860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f28ec069f30 tx=0x7f28e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:13.892 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28d3fff700 1 -- 172.21.15.99:0/3536991557 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4012430 con 0x7f28ec156f00 2022-11-27T14:35:13.892 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28d3fff700 1 -- 172.21.15.99:0/3536991557 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e4020070 con 0x7f28ec156f00 2022-11-27T14:35:13.892 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.870+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec151a60 con 0x7f28ec156f00 2022-11-27T14:35:13.893 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.871+0000 7f28d3fff700 1 -- 172.21.15.99:0/3536991557 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4004280 con 0x7f28ec156f00 2022-11-27T14:35:13.893 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.871+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec0aa910 con 0x7f28ec156f00 2022-11-27T14:35:13.893 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.871+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec1518c0 con 0x7f28ec156f00 2022-11-27T14:35:13.893 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.872+0000 7f28d3fff700 1 -- 172.21.15.99:0/3536991557 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f28e402e070 con 0x7f28ec156f00 2022-11-27T14:35:13.894 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.872+0000 7f28d3fff700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f28cc066590 0x7f28cc068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:13.894 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.872+0000 7f28d3fff700 1 -- 172.21.15.99:0/3536991557 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f28e4022070 con 0x7f28ec156f00 2022-11-27T14:35:13.894 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.872+0000 7f28eb7fe700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f28cc066590 0x7f28cc068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:13.895 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.873+0000 7f28eb7fe700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f28cc066590 0x7f28cc068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f28dc007fd0 tx=0x7f28dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:13.896 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:13.876+0000 7f28d3fff700 1 -- 172.21.15.99:0/3536991557 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f28ec1518c0 con 0x7f28ec156f00 2022-11-27T14:35:14.057 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.056+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f28ec1518c0 con 0x7f28ec156f00 2022-11-27T14:35:14.505 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.504+0000 7f28d3fff700 1 -- 172.21.15.99:0/3536991557 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f28ec1518c0 con 0x7f28ec156f00 2022-11-27T14:35:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f28cc066590 msgr2=0x7f28cc068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f28cc066590 0x7f28cc068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f28dc007fd0 tx=0x7f28dc009040 comp rx=0 tx=0).stop 2022-11-27T14:35:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec156f00 msgr2=0x7f28ec14f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:14.509 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec156f00 0x7f28ec14f860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f28ec069f30 tx=0x7f28e4004ad0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.509 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 shutdown_connections 2022-11-27T14:35:14.509 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f28cc066590 0x7f28cc068a40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.509 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28ec156f00 0x7f28ec14f860 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.510 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28ec0c4db0 0x7f28ec0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.510 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 --2- 172.21.15.99:0/3536991557 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f28ec0c6940 0x7f28ec0abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.510 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 >> 172.21.15.99:0/3536991557 conn(0x7f28ec069380 msgr2=0x7f28ec0c3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:14.510 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 shutdown_connections 2022-11-27T14:35:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.506+0000 7f28f1b2f700 1 -- 172.21.15.99:0/3536991557 wait complete. 2022-11-27T14:35:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'rbd' created 2022-11-27T14:35:14.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-27T14:35:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.642+0000 7f6c21bcf700 1 Processor -- start 2022-11-27T14:35:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.642+0000 7f6c21bcf700 1 -- start start 2022-11-27T14:35:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.642+0000 7f6c21bcf700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6c1c06e280 0x7f6c1c15e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c21bcf700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c0c1cb0 con 0x7f6c1c156fd0 2022-11-27T14:35:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c21bcf700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c0c1f90 con 0x7f6c1c0c5450 2022-11-27T14:35:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c21bcf700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c0c2270 con 0x7f6c1c06e280 2022-11-27T14:35:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1b7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6c1c156fd0 0x7f6c1c06eb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51036/0 (socket says 172.21.15.99:51036) 2022-11-27T14:35:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1b7fe700 1 -- 172.21.15.99:0/3473464133 learned_addr learned my addr 172.21.15.99:0/3473464133 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:35:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280950251 0 0) 0x7f6c1c0c1cb0 con 0x7f6c1c156fd0 2022-11-27T14:35:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c080035b0 con 0x7f6c1c156fd0 2022-11-27T14:35:14.650 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1bfff700 1 --2- 172.21.15.99:0/3473464133 >> v2:172.21.15.137:3300/0 conn(0x7f6c1c06e280 0x7f6c1c15e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:14.650 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1072410083 0 0) 0x7f6c080035b0 con 0x7f6c1c156fd0 2022-11-27T14:35:14.650 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 >> v2:172.21.15.137:3300/0 conn(0x7f6c1c06e280 msgr2=0x7f6c1c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:14.650 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 --2- 172.21.15.99:0/3473464133 >> v2:172.21.15.137:3300/0 conn(0x7f6c1c06e280 0x7f6c1c15e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.651 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 >> v1:172.21.15.137:6789/0 conn(0x7f6c1c0c5450 legacy=0x7f6c1c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:14.651 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0c2550 con 0x7f6c1c156fd0 2022-11-27T14:35:14.651 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2287561895 0 0) 0x7f6c0c001670 con 0x7f6c1c156fd0 2022-11-27T14:35:14.651 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 >> v1:172.21.15.99:6789/0 conn(0x7f6c1c156fd0 legacy=0x7f6c1c06eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:14.652 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c08003a00 0x7f6c08003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:14.652 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c08004850 0x7f6c08004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:14.652 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08005200 0x7f6c0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:14.653 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0c2270 con 0x7f6c08004850 2022-11-27T14:35:14.653 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0800c110 con 0x7f6c08005200 2022-11-27T14:35:14.653 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0800c2d0 con 0x7f6c08003a00 2022-11-27T14:35:14.653 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1affd700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c08004850 0x7f6c08004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:14.654 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1b7fe700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c08003a00 0x7f6c08003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:14.654 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.643+0000 7f6c1bfff700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08005200 0x7f6c0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:14.654 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1affd700 1 -- 172.21.15.99:0/3473464133 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c08003a00 msgr2=0x7f6c08003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:14.654 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1affd700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c08003a00 0x7f6c08003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.655 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1affd700 1 -- 172.21.15.99:0/3473464133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08005200 msgr2=0x7f6c0800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:14.655 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1affd700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08005200 0x7f6c0800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.655 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1bfff700 1 -- 172.21.15.99:0/3473464133 reap_dead start 2022-11-27T14:35:14.655 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1affd700 1 -- 172.21.15.99:0/3473464133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c080035b0 con 0x7f6c08004850 2022-11-27T14:35:14.656 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1affd700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c08004850 0x7f6c08004cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c10001800 tx=0x7f6c08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857bb7aae4c1ab3f server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:14.656 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c100012f0 con 0x7f6c08004850 2022-11-27T14:35:14.656 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c10005310 con 0x7f6c08004850 2022-11-27T14:35:14.657 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c1a7fc700 1 -- 172.21.15.99:0/3473464133 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10006860 con 0x7f6c08004850 2022-11-27T14:35:14.657 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c21bcf700 1 -- 172.21.15.99:0/3473464133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c08004850 msgr2=0x7f6c08004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:14.658 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c21bcf700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c08004850 0x7f6c08004cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c10001800 tx=0x7f6c08005880 comp rx=0 tx=0).stop 2022-11-27T14:35:14.658 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c21bcf700 1 -- 172.21.15.99:0/3473464133 shutdown_connections 2022-11-27T14:35:14.658 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c21bcf700 1 --2- 172.21.15.99:0/3473464133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c08004850 0x7f6c08004cc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.659 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c21bcf700 1 -- 172.21.15.99:0/3473464133 >> 172.21.15.99:0/3473464133 conn(0x7f6c1c069380 msgr2=0x7f6c1c069750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:14.659 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c21bcf700 1 -- 172.21.15.99:0/3473464133 shutdown_connections 2022-11-27T14:35:14.660 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.644+0000 7f6c21bcf700 1 -- 172.21.15.99:0/3473464133 wait complete. 2022-11-27T14:35:14.660 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.645+0000 7f6c21bcf700 1 Processor -- start 2022-11-27T14:35:14.661 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c21bcf700 1 -- start start 2022-11-27T14:35:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c21bcf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08004850 0x7f6c1c0a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c21bcf700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c1c152d90 0x7f6c1c0ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:14.668 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c21bcf700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c14f8a0 0x7f6c1c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:14.668 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c21bcf700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1ebce0 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.668 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c21bcf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c153200 con 0x7f6c08004850 2022-11-27T14:35:14.669 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c21bcf700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c1eb0f0 con 0x7f6c1c152d90 2022-11-27T14:35:14.669 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c14f8a0 0x7f6c1c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:14.669 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c14f8a0 0x7f6c1c14fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55078/0 (socket says 172.21.15.99:55078) 2022-11-27T14:35:14.669 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c1bfff700 1 -- 172.21.15.99:0/2702976659 learned_addr learned my addr 172.21.15.99:0/2702976659 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.646+0000 7f6c1affd700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c1c152d90 0x7f6c1c0ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c1b7fe700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08004850 0x7f6c1c0a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c1bfff700 1 -- 172.21.15.99:0/2702976659 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c1c152d90 msgr2=0x7f6c1c0ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c1bfff700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c1c152d90 0x7f6c1c0ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c1bfff700 1 -- 172.21.15.99:0/2702976659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08004850 msgr2=0x7f6c1c0a9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c1bfff700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08004850 0x7f6c1c0a9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c1bfff700 1 -- 172.21.15.99:0/2702976659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0a61e0 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c1bfff700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c14f8a0 0x7f6c1c14fd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c06c5d0 tx=0x7f6c14011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c03fff700 1 -- 172.21.15.99:0/2702976659 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400e040 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c03fff700 1 -- 172.21.15.99:0/2702976659 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1400f040 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c03fff700 1 -- 172.21.15.99:0/2702976659 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1400fc10 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c1eb340 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.647+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c1eb900 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.648+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c1ebe20 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.648+0000 7f6c03fff700 1 -- 172.21.15.99:0/2702976659 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f6c1400fd70 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.649+0000 7f6c03fff700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6bfc066590 0x7f6bfc068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.649+0000 7f6c03fff700 1 -- 172.21.15.99:0/2702976659 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f6c14083b30 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.674 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.649+0000 7f6c1b7fe700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6bfc066590 0x7f6bfc068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:14.674 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.649+0000 7f6c1b7fe700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6bfc066590 0x7f6bfc068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0a6070 tx=0x7f6c0c000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:14.674 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.653+0000 7f6c03fff700 1 -- 172.21.15.99:0/2702976659 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f6c1c1ebe20 con 0x7f6c1c14f8a0 2022-11-27T14:35:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:14.833+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6c1c1ebe20 con 0x7f6c1c14f8a0 2022-11-27T14:35:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.510+0000 7f6c03fff700 1 -- 172.21.15.99:0/2702976659 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f6c1c1ebe20 con 0x7f6c1c14f8a0 2022-11-27T14:35:15.515 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6bfc066590 msgr2=0x7f6bfc068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:15.515 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6bfc066590 0x7f6bfc068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0a6070 tx=0x7f6c0c000ed0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.516 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c14f8a0 msgr2=0x7f6c1c14fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:15.516 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c14f8a0 0x7f6c1c14fd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c06c5d0 tx=0x7f6c14011ed0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.516 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 shutdown_connections 2022-11-27T14:35:15.516 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6bfc066590 0x7f6bfc068a40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.517 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c14f8a0 0x7f6c1c14fd10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.517 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c08004850 0x7f6c1c0a9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.517 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 --2- 172.21.15.99:0/2702976659 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c1c152d90 0x7f6c1c0ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.517 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 >> 172.21.15.99:0/2702976659 conn(0x7f6c1c069380 msgr2=0x7f6c1c156210 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:15.517 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 shutdown_connections 2022-11-27T14:35:15.518 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.513+0000 7f6c21bcf700 1 -- 172.21.15.99:0/2702976659 wait complete. 2022-11-27T14:35:15.518 INFO:teuthology.orchestra.run.smithi099.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-27T14:35:15.529 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-27T14:35:15.530 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-27T14:35:15.531 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-27T14:35:15.533 INFO:tasks.ceph.mds.a:Started 2022-11-27T14:35:15.534 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-27T14:35:15.534 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-27T14:35:15.536 INFO:tasks.ceph.mds.c:Started 2022-11-27T14:35:15.537 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-27T14:35:15.537 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-27T14:35:15.540 INFO:tasks.ceph.mds.b:Started 2022-11-27T14:35:15.540 INFO:tasks.ceph.mds.d:Restarting daemon 2022-11-27T14:35:15.540 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-11-27T14:35:15.542 INFO:tasks.ceph.mds.d:Started 2022-11-27T14:35:15.543 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-27T14:35:15.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T14:35:15.616 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2022-11-27T14:35:15.616 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2022-11-27T14:35:15.624 INFO:tasks.ceph.mds.d.smithi137.stdout:starting mds.d at 2022-11-27T14:35:15.624 INFO:tasks.ceph.mds.b.smithi137.stdout:starting mds.b at 2022-11-27T14:35:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.676+0000 7fb6c9703700 1 Processor -- start 2022-11-27T14:35:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.676+0000 7fb6c9703700 1 -- start start 2022-11-27T14:35:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c9703700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6c414e400 0x7fb6c40c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c9703700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 0x7fb6c40c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c9703700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c40c4490 con 0x7fb6c4151710 2022-11-27T14:35:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c9703700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c414e7d0 con 0x7fb6c414e400 2022-11-27T14:35:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c9703700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c4150a40 con 0x7fb6c40c50e0 2022-11-27T14:35:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c2ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6c414e400 0x7fb6c40c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c2ffd700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6c414e400 0x7fb6c40c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55104/0 (socket says 172.21.15.99:55104) 2022-11-27T14:35:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c27fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb6c4151710 0x7fb6c40c0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51080/0 (socket says 172.21.15.99:51080) 2022-11-27T14:35:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.677+0000 7fb6c2ffd700 1 -- 172.21.15.99:0/3600235595 learned_addr learned my addr 172.21.15.99:0/3600235595 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c37fe700 1 --2- 172.21.15.99:0/3600235595 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 0x7fb6c40c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c2ffd700 1 -- 172.21.15.99:0/3600235595 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 msgr2=0x7fb6c40c2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c2ffd700 1 --2- 172.21.15.99:0/3600235595 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 0x7fb6c40c2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c2ffd700 1 -- 172.21.15.99:0/3600235595 >> v1:172.21.15.99:6789/0 conn(0x7fb6c4151710 legacy=0x7fb6c40c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c2ffd700 1 -- 172.21.15.99:0/3600235595 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c4150c30 con 0x7fb6c414e400 2022-11-27T14:35:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c1ffb700 1 -- 172.21.15.99:0/3600235595 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484614096 0 0) 0x7fb6c40c4490 con 0x7fb6c4151710 2022-11-27T14:35:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c2ffd700 1 --2- 172.21.15.99:0/3600235595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c414e400 0x7fb6c40c4770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80098f0 tx=0x7fb6b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7ac8d89a45dc17 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c1ffb700 1 -- 172.21.15.99:0/3600235595 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b800b8a0 con 0x7fb6c414e400 2022-11-27T14:35:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c1ffb700 1 -- 172.21.15.99:0/3600235595 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b800ba00 con 0x7fb6c414e400 2022-11-27T14:35:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c1ffb700 1 -- 172.21.15.99:0/3600235595 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b800bd90 con 0x7fb6c414e400 2022-11-27T14:35:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c9703700 1 -- 172.21.15.99:0/3600235595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c414e400 msgr2=0x7fb6c40c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3600235595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c414e400 0x7fb6c40c4770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80098f0 tx=0x7fb6b8009c00 comp rx=0 tx=0).stop 2022-11-27T14:35:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c9703700 1 -- 172.21.15.99:0/3600235595 shutdown_connections 2022-11-27T14:35:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3600235595 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 0x7fb6c40c2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3600235595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c414e400 0x7fb6c40c4770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.678+0000 7fb6c9703700 1 -- 172.21.15.99:0/3600235595 >> 172.21.15.99:0/3600235595 conn(0x7fb6c40a7760 msgr2=0x7fb6c40a7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.679+0000 7fb6c9703700 1 -- 172.21.15.99:0/3600235595 shutdown_connections 2022-11-27T14:35:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.679+0000 7fb6c9703700 1 -- 172.21.15.99:0/3600235595 wait complete. 2022-11-27T14:35:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.680+0000 7fb6c9703700 1 Processor -- start 2022-11-27T14:35:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.680+0000 7fb6c9703700 1 -- start start 2022-11-27T14:35:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.680+0000 7fb6c9703700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 0x7fb6c41db220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.680+0000 7fb6c9703700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c414e400 0x7fb6c40a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.680+0000 7fb6c9703700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4151710 0x7fb6c4144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.680+0000 7fb6c9703700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40a5e00 con 0x7fb6c4151710 2022-11-27T14:35:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.680+0000 7fb6c9703700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41e9c70 con 0x7fb6c414e400 2022-11-27T14:35:15.694 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.680+0000 7fb6c9703700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c40a6010 con 0x7fb6c40c50e0 2022-11-27T14:35:15.694 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4151710 0x7fb6c4144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:15.694 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4151710 0x7fb6c4144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55114/0 (socket says 172.21.15.99:55114) 2022-11-27T14:35:15.695 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 -- 172.21.15.99:0/3014900242 learned_addr learned my addr 172.21.15.99:0/3014900242 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:15.695 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c27fc700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c414e400 0x7fb6c40a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:15.696 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c2ffd700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 0x7fb6c41db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:15.696 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 -- 172.21.15.99:0/3014900242 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 msgr2=0x7fb6c41db220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:15.697 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 0x7fb6c41db220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.697 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 -- 172.21.15.99:0/3014900242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c414e400 msgr2=0x7fb6c40a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:15.697 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c414e400 0x7fb6c40a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.698 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 -- 172.21.15.99:0/3014900242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b8009550 con 0x7fb6c4151710 2022-11-27T14:35:15.698 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c37fe700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4151710 0x7fb6c4144e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc007e60 tx=0x7fb6bc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6ab7fe700 1 -- 172.21.15.99:0/3014900242 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc00ed30 con 0x7fb6c4151710 2022-11-27T14:35:15.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6ab7fe700 1 -- 172.21.15.99:0/3014900242 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc014420 con 0x7fb6c4151710 2022-11-27T14:35:15.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c40a62f0 con 0x7fb6c4151710 2022-11-27T14:35:15.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6ab7fe700 1 -- 172.21.15.99:0/3014900242 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0147d0 con 0x7fb6c4151710 2022-11-27T14:35:15.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.681+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c41e98f0 con 0x7fb6c4151710 2022-11-27T14:35:15.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.682+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c40a6810 con 0x7fb6c4151710 2022-11-27T14:35:15.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.682+0000 7fb6ab7fe700 1 -- 172.21.15.99:0/3014900242 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fb6bc012050 con 0x7fb6c4151710 2022-11-27T14:35:15.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.683+0000 7fb6ab7fe700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fb6a4066590 0x7fb6a4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:15.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.683+0000 7fb6c2ffd700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fb6a4066590 0x7fb6a4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:15.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.683+0000 7fb6ab7fe700 1 -- 172.21.15.99:0/3014900242 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb6bc083960 con 0x7fb6c4151710 2022-11-27T14:35:15.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.683+0000 7fb6c2ffd700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fb6a4066590 0x7fb6a4068a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8014040 tx=0x7fb6b8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:15.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.687+0000 7fb6ab7fe700 1 -- 172.21.15.99:0/3014900242 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fb6c40a6810 con 0x7fb6c4151710 2022-11-27T14:35:15.870 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.869+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6c40a6810 con 0x7fb6c4151710 2022-11-27T14:35:15.871 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.870+0000 7fb6ab7fe700 1 -- 172.21.15.99:0/3014900242 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13754 (secure 0 0 0) 0x7fb6c40a6810 con 0x7fb6c4151710 2022-11-27T14:35:15.872 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:35:15.872 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":18,"fsid":"1f55be23-348c-4a1c-92de-37e343596508","created":"2022-11-27T14:34:55.727085+0000","modified":"2022-11-27T14:35:15.503895+0000","last_up_change":"2022-11-27T14:35:06.926207+0000","last_in_change":"2022-11-27T14:34:58.198186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T14:35:07.033933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T14:35:14.075015+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":101281},{"type":"v1","addr":"172.21.15.99:6801","nonce":101281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":101281},{"type":"v1","addr":"172.21.15.99:6803","nonce":101281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":101281},{"type":"v1","addr":"172.21.15.99:6807","nonce":101281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":101281},{"type":"v1","addr":"172.21.15.99:6805","nonce":101281}]},"public_addr":"172.21.15.99:6801/101281","cluster_addr":"172.21.15.99:6803/101281","heartbeat_back_addr":"172.21.15.99:6807/101281","heartbeat_front_addr":"172.21.15.99:6805/101281","state":["exists","up"]},{"osd":1,"uuid":"6cb243f2-dcee-4f20-87da-02657be3dce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":101290},{"type":"v1","addr":"172.21.15.99:6809","nonce":101290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":101290},{"type":"v1","addr":"172.21.15.99:6811","nonce":101290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":101290},{"type":"v1","addr":"172.21.15.99:6815","nonce":101290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":101290},{"type":"v1","addr":"172.21.15.99:6813","nonce":101290}]},"public_addr":"172.21.15.99:6809/101290","cluster_addr":"172.21.15.99:6811/101290","heartbeat_back_addr":"172.21.15.99:6815/101290","heartbeat_front_addr":"172.21.15.99:6813/101290","state":["exists","up"]},{"osd":2,"uuid":"6206b34c-8c3d-4337-9f81-27efd1811bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":101291},{"type":"v1","addr":"172.21.15.99:6817","nonce":101291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":101291},{"type":"v1","addr":"172.21.15.99:6819","nonce":101291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":101291},{"type":"v1","addr":"172.21.15.99:6823","nonce":101291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":101291},{"type":"v1","addr":"172.21.15.99:6821","nonce":101291}]},"public_addr":"172.21.15.99:6817/101291","cluster_addr":"172.21.15.99:6819/101291","heartbeat_back_addr":"172.21.15.99:6823/101291","heartbeat_front_addr":"172.21.15.99:6821/101291","state":["exists","up"]},{"osd":3,"uuid":"b7d6bc6e-1be5-4355-8c30-6ae75fc573e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":101298},{"type":"v1","addr":"172.21.15.99:6825","nonce":101298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":101298},{"type":"v1","addr":"172.21.15.99:6827","nonce":101298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":101298},{"type":"v1","addr":"172.21.15.99:6831","nonce":101298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":101298},{"type":"v1","addr":"172.21.15.99:6829","nonce":101298}]},"public_addr":"172.21.15.99:6825/101298","cluster_addr":"172.21.15.99:6827/101298","heartbeat_back_addr":"172.21.15.99:6831/101298","heartbeat_front_addr":"172.21.15.99:6829/101298","state":["exists","up"]},{"osd":4,"uuid":"983ec37a-caef-4592-81b1-27347e23ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":101499},{"type":"v1","addr":"172.21.15.137:6818","nonce":101499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":101499},{"type":"v1","addr":"172.21.15.137:6822","nonce":101499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":101499},{"type":"v1","addr":"172.21.15.137:6830","nonce":101499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":101499},{"type":"v1","addr":"172.21.15.137:6826","nonce":101499}]},"public_addr":"172.21.15.137:6818/101499","cluster_addr":"172.21.15.137:6822/101499","heartbeat_back_addr":"172.21.15.137:6830/101499","heartbeat_front_addr":"172.21.15.137:6826/101499","state":["exists","up"]},{"osd":5,"uuid":"a7625e4c-0f53-4600-be02-c6d0268818bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":101475},{"type":"v1","addr":"172.21.15.137:6801","nonce":101475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":101475},{"type":"v1","addr":"172.21.15.137:6803","nonce":101475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":101475},{"type":"v1","addr":"172.21.15.137:6807","nonce":101475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":101475},{"type":"v1","addr":"172.21.15.137:6805","nonce":101475}]},"public_addr":"172.21.15.137:6801/101475","cluster_addr":"172.21.15.137:6803/101475","heartbeat_back_addr":"172.21.15.137:6807/101475","heartbeat_front_addr":"172.21.15.137:6805/101475","state":["exists","up"]},{"osd":6,"uuid":"c06a48ca-e3d8-45e1-b94f-9a391541ba8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":101476},{"type":"v1","addr":"172.21.15.137:6809","nonce":101476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":101476},{"type":"v1","addr":"172.21.15.137:6811","nonce":101476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":101476},{"type":"v1","addr":"172.21.15.137:6815","nonce":101476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":101476},{"type":"v1","addr":"172.21.15.137:6813","nonce":101476}]},"public_addr":"172.21.15.137:6809/101476","cluster_addr":"172.21.15.137:6811/101476","heartbeat_back_addr":"172.21.15.137:6815/101476","heartbeat_front_addr":"172.21.15.137:6813/101476","state":["exists","up"]},{"osd":7,"uuid":"2302634c-0ed5-4467-ac3c-805b93519b9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6817","nonce":101506},{"type":"v1","addr":"172.21.15.137:6819","nonce":101506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6821","nonce":101506},{"type":"v1","addr":"172.21.15.137:6823","nonce":101506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6829","nonce":101506},{"type":"v1","addr":"172.21.15.137:6831","nonce":101506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":101506},{"type":"v1","addr":"172.21.15.137:6827","nonce":101506}]},"public_addr":"172.21.15.137:6819/101506","cluster_addr":"172.21.15.137:6823/101506","heartbeat_back_addr":"172.21.15.137:6831/101506","heartbeat_front_addr":"172.21.15.137:6827/101506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.371397+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.413296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.430729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.359936+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.443571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.389589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.404518+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.449713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T14:35:15.874 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fb6a4066590 msgr2=0x7fb6a4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:15.874 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fb6a4066590 0x7fb6a4068a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8014040 tx=0x7fb6b8008030 comp rx=0 tx=0).stop 2022-11-27T14:35:15.874 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4151710 msgr2=0x7fb6c4144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:15.874 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4151710 0x7fb6c4144e70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc007e60 tx=0x7fb6bc00e3c0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.875 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 shutdown_connections 2022-11-27T14:35:15.875 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7fb6a4066590 0x7fb6a4068a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.875 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6c4151710 0x7fb6c4144e70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.875 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7fb6c40c50e0 0x7fb6c41db220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.876 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 --2- 172.21.15.99:0/3014900242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6c414e400 0x7fb6c40a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:15.876 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 >> 172.21.15.99:0/3014900242 conn(0x7fb6c40a7760 msgr2=0x7fb6c4071330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:15.876 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 shutdown_connections 2022-11-27T14:35:15.876 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:15.872+0000 7fb6c9703700 1 -- 172.21.15.99:0/3014900242 wait complete. 2022-11-27T14:35:15.886 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T14:35:07.033933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T14:35:14.075015+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T14:35:15.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T14:35:16.012 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.008+0000 7ff25aa53700 1 Processor -- start 2022-11-27T14:35:16.012 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.008+0000 7ff25aa53700 1 -- start start 2022-11-27T14:35:16.013 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff25aa53700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2540c5480 0x7ff2540c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.013 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff25aa53700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff254150cd0 con 0x7ff254152da0 2022-11-27T14:35:16.013 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff25aa53700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff254071500 con 0x7ff25414ce60 2022-11-27T14:35:16.013 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff25aa53700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff254153c20 con 0x7ff2540c5480 2022-11-27T14:35:16.014 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff2537fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2540c5480 0x7ff2540c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.014 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff258ff0700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff254152da0 0x7ff25415a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51096/0 (socket says 172.21.15.99:51096) 2022-11-27T14:35:16.014 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff258ff0700 1 -- 172.21.15.99:0/3151199227 learned_addr learned my addr 172.21.15.99:0/3151199227 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:35:16.014 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff2537fe700 1 -- 172.21.15.99:0/3151199227 >> v1:172.21.15.137:6789/0 conn(0x7ff25414ce60 legacy=0x7ff25406fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.015 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff2537fe700 1 -- 172.21.15.99:0/3151199227 >> v1:172.21.15.99:6789/0 conn(0x7ff254152da0 legacy=0x7ff25415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.015 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff2537fe700 1 -- 172.21.15.99:0/3151199227 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25414d230 con 0x7ff2540c5480 2022-11-27T14:35:16.015 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff252ffd700 1 -- 172.21.15.99:0/3151199227 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689939521 0 0) 0x7ff254150cd0 con 0x7ff254152da0 2022-11-27T14:35:16.015 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.009+0000 7ff2537fe700 1 --2- 172.21.15.99:0/3151199227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540c5480 0x7ff2540c44f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff248009840 tx=0x7ff248009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a7ac51c5a61c8b server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff252ffd700 1 -- 172.21.15.99:0/3151199227 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248011830 con 0x7ff2540c5480 2022-11-27T14:35:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff252ffd700 1 -- 172.21.15.99:0/3151199227 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff248011990 con 0x7ff2540c5480 2022-11-27T14:35:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff252ffd700 1 -- 172.21.15.99:0/3151199227 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248011d20 con 0x7ff2540c5480 2022-11-27T14:35:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff25aa53700 1 -- 172.21.15.99:0/3151199227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540c5480 msgr2=0x7ff2540c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.016 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff25aa53700 1 --2- 172.21.15.99:0/3151199227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540c5480 0x7ff2540c44f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff248009840 tx=0x7ff248009b50 comp rx=0 tx=0).stop 2022-11-27T14:35:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff25aa53700 1 -- 172.21.15.99:0/3151199227 shutdown_connections 2022-11-27T14:35:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff25aa53700 1 --2- 172.21.15.99:0/3151199227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2540c5480 0x7ff2540c44f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff25aa53700 1 -- 172.21.15.99:0/3151199227 >> 172.21.15.99:0/3151199227 conn(0x7ff2540a77f0 msgr2=0x7ff2540a7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:16.017 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff25aa53700 1 -- 172.21.15.99:0/3151199227 shutdown_connections 2022-11-27T14:35:16.018 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.010+0000 7ff25aa53700 1 -- 172.21.15.99:0/3151199227 wait complete. 2022-11-27T14:35:16.021 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.011+0000 7ff25aa53700 1 Processor -- start 2022-11-27T14:35:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.011+0000 7ff25aa53700 1 -- start start 2022-11-27T14:35:16.022 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff25aa53700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2540c5480 0x7ff2541eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff25aa53700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25414ce60 0x7ff254148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.023 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff25aa53700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff254152da0 0x7ff2541e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.024 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff25aa53700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25406d630 con 0x7ff25414ce60 2022-11-27T14:35:16.024 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff25aa53700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25414a840 con 0x7ff2540c5480 2022-11-27T14:35:16.024 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff25aa53700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2541eb7c0 con 0x7ff254152da0 2022-11-27T14:35:16.025 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff2537fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25414ce60 0x7ff254148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.025 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff2537fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25414ce60 0x7ff254148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55126/0 (socket says 172.21.15.99:55126) 2022-11-27T14:35:16.026 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff2537fe700 1 -- 172.21.15.99:0/4190296119 learned_addr learned my addr 172.21.15.99:0/4190296119 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:16.026 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff258ff0700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff254152da0 0x7ff2541e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.027 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.012+0000 7ff253fff700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2540c5480 0x7ff2541eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.027 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff2537fe700 1 -- 172.21.15.99:0/4190296119 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff254152da0 msgr2=0x7ff2541e82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.027 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff2537fe700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff254152da0 0x7ff2541e82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.028 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff2537fe700 1 -- 172.21.15.99:0/4190296119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2540c5480 msgr2=0x7ff2541eb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.028 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff2537fe700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2540c5480 0x7ff2541eb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.029 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff2537fe700 1 -- 172.21.15.99:0/4190296119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2480094a0 con 0x7ff25414ce60 2022-11-27T14:35:16.029 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff2537fe700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25414ce60 0x7ff254148400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2480090b0 tx=0x7ff248004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.029 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff250ff9700 1 -- 172.21.15.99:0/4190296119 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24801a7e0 con 0x7ff25414ce60 2022-11-27T14:35:16.030 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2541ebaa0 con 0x7ff25414ce60 2022-11-27T14:35:16.030 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff250ff9700 1 -- 172.21.15.99:0/4190296119 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24801a940 con 0x7ff25414ce60 2022-11-27T14:35:16.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff250ff9700 1 -- 172.21.15.99:0/4190296119 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24801acb0 con 0x7ff25414ce60 2022-11-27T14:35:16.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.013+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2541ec060 con 0x7ff25414ce60 2022-11-27T14:35:16.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.014+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2540733f0 con 0x7ff25414ce60 2022-11-27T14:35:16.032 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.014+0000 7ff250ff9700 1 -- 172.21.15.99:0/4190296119 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7ff24800f050 con 0x7ff25414ce60 2022-11-27T14:35:16.032 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.014+0000 7ff250ff9700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7ff234066590 0x7ff234068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.032 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.015+0000 7ff253fff700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7ff234066590 0x7ff234068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.033 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.015+0000 7ff250ff9700 1 -- 172.21.15.99:0/4190296119 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff2480843b0 con 0x7ff25414ce60 2022-11-27T14:35:16.033 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.015+0000 7ff253fff700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7ff234066590 0x7ff234068a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff2540723c0 tx=0x7ff244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.033 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.019+0000 7ff250ff9700 1 -- 172.21.15.99:0/4190296119 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7ff248051e40 con 0x7ff25414ce60 2022-11-27T14:35:16.197 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.195+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2540c5260 con 0x7ff25414ce60 2022-11-27T14:35:16.198 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.196+0000 7ff250ff9700 1 -- 172.21.15.99:0/4190296119 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff248015020 con 0x7ff25414ce60 2022-11-27T14:35:16.199 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2022-11-27T14:35:16.201 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7ff234066590 msgr2=0x7ff234068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.202 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7ff234066590 0x7ff234068a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff2540723c0 tx=0x7ff244009040 comp rx=0 tx=0).stop 2022-11-27T14:35:16.202 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25414ce60 msgr2=0x7ff254148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.203 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25414ce60 0x7ff254148400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2480090b0 tx=0x7ff248004750 comp rx=0 tx=0).stop 2022-11-27T14:35:16.203 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 shutdown_connections 2022-11-27T14:35:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7ff234066590 0x7ff234068a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7ff254152da0 0x7ff2541e82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.204 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2540c5480 0x7ff2541eb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 --2- 172.21.15.99:0/4190296119 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff25414ce60 0x7ff254148400 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.205 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 >> 172.21.15.99:0/4190296119 conn(0x7ff2540a77f0 msgr2=0x7ff2540a7030 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 shutdown_connections 2022-11-27T14:35:16.206 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.198+0000 7ff25aa53700 1 -- 172.21.15.99:0/4190296119 wait complete. 2022-11-27T14:35:16.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T14:35:16.335 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.333+0000 7f87b660b700 1 Processor -- start 2022-11-27T14:35:16.335 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.333+0000 7f87b660b700 1 -- start start 2022-11-27T14:35:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87b660b700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87b00bf820 0x7f87b01534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.338 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87b660b700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 0x7f87b01565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.339 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87b660b700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b014cf70 con 0x7f87b014cba0 2022-11-27T14:35:16.339 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87b660b700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b00c0630 con 0x7f87b00bf820 2022-11-27T14:35:16.339 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87b660b700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0153a30 con 0x7f87b0156140 2022-11-27T14:35:16.339 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87af7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87b00bf820 0x7f87b01534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.340 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87af7fe700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87b00bf820 0x7f87b01534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55134/0 (socket says 172.21.15.99:55134) 2022-11-27T14:35:16.340 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87af7fe700 1 -- 172.21.15.99:0/339249803 learned_addr learned my addr 172.21.15.99:0/339249803 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:16.340 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.334+0000 7f87b4ba8700 1 --2- 172.21.15.99:0/339249803 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 0x7f87b01565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.340 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87af7fe700 1 -- 172.21.15.99:0/339249803 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 msgr2=0x7f87b01565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:16.341 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87af7fe700 1 --2- 172.21.15.99:0/339249803 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 0x7f87b01565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.341 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87af7fe700 1 -- 172.21.15.99:0/339249803 >> v1:172.21.15.137:6789/0 conn(0x7f87b014cba0 legacy=0x7f87b0069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.341 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87af7fe700 1 -- 172.21.15.99:0/339249803 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b00a4f20 con 0x7f87b00bf820 2022-11-27T14:35:16.341 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87af7fe700 1 --2- 172.21.15.99:0/339249803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b01534f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009920 tx=0x7f87a0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638f2ac08a448b83 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.342 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87aeffd700 1 -- 172.21.15.99:0/339249803 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000ba20 con 0x7f87b00bf820 2022-11-27T14:35:16.342 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87aeffd700 1 -- 172.21.15.99:0/339249803 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a000bb80 con 0x7f87b00bf820 2022-11-27T14:35:16.342 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87aeffd700 1 -- 172.21.15.99:0/339249803 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0009e70 con 0x7f87b00bf820 2022-11-27T14:35:16.342 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87b660b700 1 -- 172.21.15.99:0/339249803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 msgr2=0x7f87b01534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.343 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87b660b700 1 --2- 172.21.15.99:0/339249803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b01534f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009920 tx=0x7f87a0009c30 comp rx=0 tx=0).stop 2022-11-27T14:35:16.343 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87b660b700 1 -- 172.21.15.99:0/339249803 shutdown_connections 2022-11-27T14:35:16.343 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87b660b700 1 --2- 172.21.15.99:0/339249803 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 0x7f87b01565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.343 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87b660b700 1 --2- 172.21.15.99:0/339249803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b01534f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.343 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.335+0000 7f87b660b700 1 -- 172.21.15.99:0/339249803 >> 172.21.15.99:0/339249803 conn(0x7f87b00a77f0 msgr2=0x7f87b00a7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:16.344 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.336+0000 7f87b660b700 1 -- 172.21.15.99:0/339249803 shutdown_connections 2022-11-27T14:35:16.344 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.336+0000 7f87b660b700 1 -- 172.21.15.99:0/339249803 wait complete. 2022-11-27T14:35:16.344 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.337+0000 7f87b660b700 1 Processor -- start 2022-11-27T14:35:16.344 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.337+0000 7f87b660b700 1 -- start start 2022-11-27T14:35:16.348 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.337+0000 7f87b660b700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b00a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.348 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.337+0000 7f87b660b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87b014cba0 0x7f87b014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.348 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.337+0000 7f87b660b700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 0x7f87b014f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.348 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.337+0000 7f87b660b700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00a5720 con 0x7f87b00bf820 2022-11-27T14:35:16.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.337+0000 7f87b660b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00a6260 con 0x7f87b014cba0 2022-11-27T14:35:16.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.337+0000 7f87b660b700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b006d150 con 0x7f87b0156140 2022-11-27T14:35:16.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b00a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b00a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55140/0 (socket says 172.21.15.99:55140) 2022-11-27T14:35:16.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 -- 172.21.15.99:0/310017891 learned_addr learned my addr 172.21.15.99:0/310017891 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:16.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 -- 172.21.15.99:0/310017891 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 msgr2=0x7f87b014f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 0x7f87b014f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 -- 172.21.15.99:0/310017891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87b014cba0 msgr2=0x7f87b014f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87b014cba0 0x7f87b014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.351 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 -- 172.21.15.99:0/310017891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0009550 con 0x7f87b00bf820 2022-11-27T14:35:16.351 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87affff700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b00a6e70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c510 tx=0x7f87a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.351 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87acff9700 1 -- 172.21.15.99:0/310017891 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40052d0 con 0x7f87b00bf820 2022-11-27T14:35:16.351 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87acff9700 1 -- 172.21.15.99:0/310017891 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a4005430 con 0x7f87b00bf820 2022-11-27T14:35:16.351 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b00a5930 con 0x7f87b00bf820 2022-11-27T14:35:16.352 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87acff9700 1 -- 172.21.15.99:0/310017891 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40038c0 con 0x7f87b00bf820 2022-11-27T14:35:16.352 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.338+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b00a5ef0 con 0x7f87b00bf820 2022-11-27T14:35:16.352 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.339+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b00733f0 con 0x7f87b00bf820 2022-11-27T14:35:16.352 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.339+0000 7f87acff9700 1 -- 172.21.15.99:0/310017891 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f87a4019030 con 0x7f87b00bf820 2022-11-27T14:35:16.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.340+0000 7f87acff9700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8790066590 0x7f8790068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.340+0000 7f87af7fe700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8790066590 0x7f8790068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.340+0000 7f87acff9700 1 -- 172.21.15.99:0/310017891 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f87a4082810 con 0x7f87b00bf820 2022-11-27T14:35:16.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.340+0000 7f87af7fe700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8790066590 0x7f8790068a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87b01560f0 tx=0x7f87a0010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.344+0000 7f87acff9700 1 -- 172.21.15.99:0/310017891 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f87a4050270 con 0x7f87b00bf820 2022-11-27T14:35:16.526 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.523+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87b00a51e0 con 0x7f87b00bf820 2022-11-27T14:35:16.526 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.524+0000 7f87acff9700 1 -- 172.21.15.99:0/310017891 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87a4017020 con 0x7f87b00bf820 2022-11-27T14:35:16.527 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 8 2022-11-27T14:35:16.529 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8790066590 msgr2=0x7f8790068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.530 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8790066590 0x7f8790068a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87b01560f0 tx=0x7f87a0010000 comp rx=0 tx=0).stop 2022-11-27T14:35:16.530 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 msgr2=0x7f87b00a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.530 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b00a6e70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c510 tx=0x7f87a400c8d0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.531 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 shutdown_connections 2022-11-27T14:35:16.531 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8790066590 0x7f8790068a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.532 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f87b0156140 0x7f87b014f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.532 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87b00bf820 0x7f87b00a6e70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.533 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 --2- 172.21.15.99:0/310017891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f87b014cba0 0x7f87b014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.533 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 >> 172.21.15.99:0/310017891 conn(0x7f87b00a77f0 msgr2=0x7f87b01565c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:16.534 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 shutdown_connections 2022-11-27T14:35:16.534 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.526+0000 7f87b660b700 1 -- 172.21.15.99:0/310017891 wait complete. 2022-11-27T14:35:16.540 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T14:35:16.663 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.661+0000 7f76b7a7a700 1 Processor -- start 2022-11-27T14:35:16.663 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.661+0000 7f76b7a7a700 1 -- start start 2022-11-27T14:35:16.670 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.662+0000 7f76b7a7a700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f76b0153280 0x7f76b0069480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.671 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.662+0000 7f76b7a7a700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b0156dd0 con 0x7f76b00c6490 2022-11-27T14:35:16.671 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.662+0000 7f76b7a7a700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b00c5220 con 0x7f76b01502a0 2022-11-27T14:35:16.671 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.662+0000 7f76b7a7a700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b0069d20 con 0x7f76b0153280 2022-11-27T14:35:16.672 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.662+0000 7f76b5015700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f76b00c6490 0x7f76b0152b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51126/0 (socket says 172.21.15.99:51126) 2022-11-27T14:35:16.673 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.662+0000 7f76b5015700 1 -- 172.21.15.99:0/657663487 learned_addr learned my addr 172.21.15.99:0/657663487 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:35:16.673 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.662+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926952878 0 0) 0x7f76b0156dd0 con 0x7f76b00c6490 2022-11-27T14:35:16.673 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.662+0000 7f76b6017700 1 --2- 172.21.15.99:0/657663487 >> v2:172.21.15.137:3300/0 conn(0x7f76b0153280 0x7f76b0069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.674 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f769c0035b0 con 0x7f76b00c6490 2022-11-27T14:35:16.674 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2322321745 0 0) 0x7f769c0035b0 con 0x7f76b00c6490 2022-11-27T14:35:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 >> v2:172.21.15.137:3300/0 conn(0x7f76b0153280 msgr2=0x7f76b0069480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 --2- 172.21.15.99:0/657663487 >> v2:172.21.15.137:3300/0 conn(0x7f76b0153280 0x7f76b0069480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 >> v1:172.21.15.137:6789/0 conn(0x7f76b01502a0 legacy=0x7f76b006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b00c5500 con 0x7f76b00c6490 2022-11-27T14:35:16.677 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2287561895 0 0) 0x7f76a4001670 con 0x7f76b00c6490 2022-11-27T14:35:16.677 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 >> v1:172.21.15.99:6789/0 conn(0x7f76b00c6490 legacy=0x7f76b0152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.678 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f769c003a00 0x7f769c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.678 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f769c004850 0x7f769c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.679 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c005200 0x7f769c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.679 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c00c0e0 con 0x7f769c003a00 2022-11-27T14:35:16.679 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c00c250 con 0x7f769c005200 2022-11-27T14:35:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c00c410 con 0x7f769c004850 2022-11-27T14:35:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b5816700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f769c003a00 0x7f769c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b5015700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f769c004850 0x7f769c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b6017700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c005200 0x7f769c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b5816700 1 -- 172.21.15.99:0/657663487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f769c004850 msgr2=0x7f769c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b5816700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f769c004850 0x7f769c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b5816700 1 -- 172.21.15.99:0/657663487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c005200 msgr2=0x7f769c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b5816700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c005200 0x7f769c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b6017700 1 -- 172.21.15.99:0/657663487 reap_dead start 2022-11-27T14:35:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b5816700 1 -- 172.21.15.99:0/657663487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0035b0 con 0x7f769c003a00 2022-11-27T14:35:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b5816700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f769c003a00 0x7f769c003e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c190 tx=0x7f769c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ca2883372e789b server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0001db0 con 0x7f769c003a00 2022-11-27T14:35:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a000ccd0 con 0x7f769c003a00 2022-11-27T14:35:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.663+0000 7f76b4814700 1 -- 172.21.15.99:0/657663487 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0003c40 con 0x7f769c003a00 2022-11-27T14:35:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.664+0000 7f76b7a7a700 1 -- 172.21.15.99:0/657663487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f769c003a00 msgr2=0x7f769c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.664+0000 7f76b7a7a700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f769c003a00 0x7f769c003e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c190 tx=0x7f769c005880 comp rx=0 tx=0).stop 2022-11-27T14:35:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.664+0000 7f76b7a7a700 1 -- 172.21.15.99:0/657663487 shutdown_connections 2022-11-27T14:35:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.664+0000 7f76b7a7a700 1 --2- 172.21.15.99:0/657663487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f769c003a00 0x7f769c003e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.664+0000 7f76b7a7a700 1 -- 172.21.15.99:0/657663487 >> 172.21.15.99:0/657663487 conn(0x7f76b00c99d0 msgr2=0x7f76b00c57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:16.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.664+0000 7f76b7a7a700 1 -- 172.21.15.99:0/657663487 shutdown_connections 2022-11-27T14:35:16.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.664+0000 7f76b7a7a700 1 -- 172.21.15.99:0/657663487 wait complete. 2022-11-27T14:35:16.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.665+0000 7f76b7a7a700 1 Processor -- start 2022-11-27T14:35:16.686 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.665+0000 7f76b7a7a700 1 -- start start 2022-11-27T14:35:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b7a7a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c003a00 0x7f76b00ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b7a7a700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76b00ace80 0x7f76b01509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b7a7a700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b0150f20 0x7f76b01eb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b7a7a700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00abb70 con 0x7f76b0150f20 2022-11-27T14:35:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b7a7a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0152880 con 0x7f769c003a00 2022-11-27T14:35:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b7a7a700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00aa5d0 con 0x7f76b00ace80 2022-11-27T14:35:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b0150f20 0x7f76b01eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b0150f20 0x7f76b01eb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55166/0 (socket says 172.21.15.99:55166) 2022-11-27T14:35:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 -- 172.21.15.99:0/1554194218 learned_addr learned my addr 172.21.15.99:0/1554194218 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 -- 172.21.15.99:0/1554194218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76b00ace80 msgr2=0x7f76b01509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.690 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b5816700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c003a00 0x7f76b00ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.690 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b5015700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76b00ace80 0x7f76b01509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.690 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76b00ace80 0x7f76b01509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.690 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 -- 172.21.15.99:0/1554194218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c003a00 msgr2=0x7f76b00ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.690 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c003a00 0x7f76b00ac940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 -- 172.21.15.99:0/1554194218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0152a80 con 0x7f76b0150f20 2022-11-27T14:35:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b6017700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b0150f20 0x7f76b01eb040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76b0153ca0 tx=0x7f76a8010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76ae7fc700 1 -- 172.21.15.99:0/1554194218 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004300 con 0x7f76b0150f20 2022-11-27T14:35:16.691 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76ae7fc700 1 -- 172.21.15.99:0/1554194218 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a8004460 con 0x7f76b0150f20 2022-11-27T14:35:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b00ae870 con 0x7f76b0150f20 2022-11-27T14:35:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.666+0000 7f76ae7fc700 1 -- 172.21.15.99:0/1554194218 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80038e0 con 0x7f76b0150f20 2022-11-27T14:35:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.667+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b00aa0b0 con 0x7f76b0150f20 2022-11-27T14:35:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.667+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b00aeab0 con 0x7f76b0150f20 2022-11-27T14:35:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.668+0000 7f76ae7fc700 1 -- 172.21.15.99:0/1554194218 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f76a8003a40 con 0x7f76b0150f20 2022-11-27T14:35:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.668+0000 7f76ae7fc700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f7690066590 0x7f7690068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.668+0000 7f76ae7fc700 1 -- 172.21.15.99:0/1554194218 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f76a80832c0 con 0x7f76b0150f20 2022-11-27T14:35:16.693 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.668+0000 7f76b5816700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f7690066590 0x7f7690068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.668+0000 7f76b5816700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f7690066590 0x7f7690068a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005040 tx=0x7f76a0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:16.694 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.672+0000 7f76ae7fc700 1 -- 172.21.15.99:0/1554194218 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f76b00aeab0 con 0x7f76b0150f20 2022-11-27T14:35:16.894 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.893+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76b00aeab0 con 0x7f76b0150f20 2022-11-27T14:35:16.895 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.893+0000 7f76ae7fc700 1 -- 172.21.15.99:0/1554194218 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f76b00aeab0 con 0x7f76b0150f20 2022-11-27T14:35:16.895 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:35:16.896 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/663264293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6833","nonce":663264293},{"type":"v1","addr":"172.21.15.137:6835","nonce":663264293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3768723858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3768723858},{"type":"v1","addr":"172.21.15.99:6836","nonce":3768723858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1684372276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1684372276},{"type":"v1","addr":"172.21.15.99:6837","nonce":1684372276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6834/3325912821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3325912821},{"type":"v1","addr":"172.21.15.137:6834","nonce":3325912821}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-11-27T14:35:16.897 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f7690066590 msgr2=0x7f7690068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.897 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f7690066590 0x7f7690068a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005040 tx=0x7f76a0009030 comp rx=0 tx=0).stop 2022-11-27T14:35:16.898 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b0150f20 msgr2=0x7f76b01eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:16.898 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b0150f20 0x7f76b01eb040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76b0153ca0 tx=0x7f76a8010b80 comp rx=0 tx=0).stop 2022-11-27T14:35:16.898 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 shutdown_connections 2022-11-27T14:35:16.899 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f7690066590 0x7f7690068a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.899 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76b0150f20 0x7f76b01eb040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.899 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f769c003a00 0x7f76b00ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.899 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 --2- 172.21.15.99:0/1554194218 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f76b00ace80 0x7f76b01509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:16.900 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.895+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 >> 172.21.15.99:0/1554194218 conn(0x7f76b00c99d0 msgr2=0x7f76b00c3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:16.900 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.896+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 shutdown_connections 2022-11-27T14:35:16.900 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:16.896+0000 7f76b7a7a700 1 -- 172.21.15.99:0/1554194218 wait complete. 2022-11-27T14:35:16.901 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 3 2022-11-27T14:35:16.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T14:35:17.030 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.028+0000 7f2e3bdca700 1 Processor -- start 2022-11-27T14:35:17.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.028+0000 7f2e3bdca700 1 -- start start 2022-11-27T14:35:17.036 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.029+0000 7f2e3bdca700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2e340bf820 0x7f2e341534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.037 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.029+0000 7f2e3bdca700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3414cf70 con 0x7f2e34156140 2022-11-27T14:35:17.037 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.029+0000 7f2e3bdca700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e340a4f20 con 0x7f2e3414cba0 2022-11-27T14:35:17.037 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.029+0000 7f2e3bdca700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e340c0630 con 0x7f2e340bf820 2022-11-27T14:35:17.037 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.029+0000 7f2e39365700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2e340bf820 0x7f2e341534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.038 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.029+0000 7f2e39365700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2e340bf820 0x7f2e341534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55174/0 (socket says 172.21.15.99:55174) 2022-11-27T14:35:17.038 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.029+0000 7f2e39365700 1 -- 172.21.15.99:0/3227097021 learned_addr learned my addr 172.21.15.99:0/3227097021 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:17.038 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.029+0000 7f2e3a367700 1 --1- 172.21.15.99:0/3227097021 >> v1:172.21.15.99:6789/0 conn(0x7f2e34156140 0x7f2e34156570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51146/0 (socket says 172.21.15.99:51146) 2022-11-27T14:35:17.038 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e39365700 1 -- 172.21.15.99:0/3227097021 >> v1:172.21.15.137:6789/0 conn(0x7f2e3414cba0 legacy=0x7f2e34069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:17.039 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e39365700 1 -- 172.21.15.99:0/3227097021 >> v1:172.21.15.99:6789/0 conn(0x7f2e34156140 legacy=0x7f2e34156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.039 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e39365700 1 -- 172.21.15.99:0/3227097021 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340a51e0 con 0x7f2e340bf820 2022-11-27T14:35:17.039 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e38b64700 1 -- 172.21.15.99:0/3227097021 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355952037 0 0) 0x7f2e3414cf70 con 0x7f2e34156140 2022-11-27T14:35:17.040 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e39365700 1 --2- 172.21.15.99:0/3227097021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e340bf820 0x7f2e341534f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009920 tx=0x7f2e24009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2114ed09458e1bc server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.040 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e38b64700 1 -- 172.21.15.99:0/3227097021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24004230 con 0x7f2e340bf820 2022-11-27T14:35:17.040 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e38b64700 1 -- 172.21.15.99:0/3227097021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e24004390 con 0x7f2e340bf820 2022-11-27T14:35:17.040 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e38b64700 1 -- 172.21.15.99:0/3227097021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24009e70 con 0x7f2e340bf820 2022-11-27T14:35:17.041 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e3bdca700 1 -- 172.21.15.99:0/3227097021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e340bf820 msgr2=0x7f2e341534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.041 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e3bdca700 1 --2- 172.21.15.99:0/3227097021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e340bf820 0x7f2e341534f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009920 tx=0x7f2e24009c30 comp rx=0 tx=0).stop 2022-11-27T14:35:17.041 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e3bdca700 1 -- 172.21.15.99:0/3227097021 shutdown_connections 2022-11-27T14:35:17.042 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e3bdca700 1 --2- 172.21.15.99:0/3227097021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e340bf820 0x7f2e341534f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.042 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e3bdca700 1 -- 172.21.15.99:0/3227097021 >> 172.21.15.99:0/3227097021 conn(0x7f2e340a77f0 msgr2=0x7f2e340a7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:17.042 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e3bdca700 1 -- 172.21.15.99:0/3227097021 shutdown_connections 2022-11-27T14:35:17.042 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.030+0000 7f2e3bdca700 1 -- 172.21.15.99:0/3227097021 wait complete. 2022-11-27T14:35:17.043 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.032+0000 7f2e3bdca700 1 Processor -- start 2022-11-27T14:35:17.043 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.032+0000 7f2e3bdca700 1 -- start start 2022-11-27T14:35:17.043 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3bdca700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2e340bf820 0x7f2e34152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.043 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3bdca700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e3414cba0 0x7f2e341eb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.044 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3bdca700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e34156140 0x7f2e340a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.044 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3bdca700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e34151330 con 0x7f2e34156140 2022-11-27T14:35:17.044 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3bdca700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e340a74d0 con 0x7f2e3414cba0 2022-11-27T14:35:17.044 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3bdca700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e341506b0 con 0x7f2e340bf820 2022-11-27T14:35:17.045 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e34156140 0x7f2e340a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.045 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e34156140 0x7f2e340a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55180/0 (socket says 172.21.15.99:55180) 2022-11-27T14:35:17.045 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 -- 172.21.15.99:0/2138203016 learned_addr learned my addr 172.21.15.99:0/2138203016 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:17.045 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e39b66700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2e340bf820 0x7f2e34152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.045 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e39365700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e3414cba0 0x7f2e341eb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.046 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 -- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2e340bf820 msgr2=0x7f2e34152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.046 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2e340bf820 0x7f2e34152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.046 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 -- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e3414cba0 msgr2=0x7f2e341eb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.046 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e3414cba0 0x7f2e341eb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.047 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 -- 172.21.15.99:0/2138203016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24009550 con 0x7f2e34156140 2022-11-27T14:35:17.047 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e39b66700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2e340bf820 0x7f2e34152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T14:35:17.047 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3a367700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e34156140 0x7f2e340a6a80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00cb60 tx=0x7f2e2c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.047 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e227fc700 1 -- 172.21.15.99:0/2138203016 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c00ab50 con 0x7f2e34156140 2022-11-27T14:35:17.048 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e227fc700 1 -- 172.21.15.99:0/2138203016 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c00acb0 con 0x7f2e34156140 2022-11-27T14:35:17.048 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e34150990 con 0x7f2e34156140 2022-11-27T14:35:17.048 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e227fc700 1 -- 172.21.15.99:0/2138203016 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c007310 con 0x7f2e34156140 2022-11-27T14:35:17.048 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.033+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3406be20 con 0x7f2e34156140 2022-11-27T14:35:17.049 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.034+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e34150eb0 con 0x7f2e34156140 2022-11-27T14:35:17.049 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.035+0000 7f2e227fc700 1 -- 172.21.15.99:0/2138203016 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f2e2c01a030 con 0x7f2e34156140 2022-11-27T14:35:17.051 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.035+0000 7f2e227fc700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f2e14066590 0x7f2e14068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.051 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.036+0000 7f2e39b66700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f2e14066590 0x7f2e14068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.051 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.036+0000 7f2e227fc700 1 -- 172.21.15.99:0/2138203016 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2e2c0832a0 con 0x7f2e34156140 2022-11-27T14:35:17.051 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.036+0000 7f2e39b66700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f2e14066590 0x7f2e14068a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009930 tx=0x7f2e30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.052 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.039+0000 7f2e227fc700 1 -- 172.21.15.99:0/2138203016 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f2e2c050da0 con 0x7f2e34156140 2022-11-27T14:35:17.222 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.220+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e3406bc80 con 0x7f2e34156140 2022-11-27T14:35:17.223 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.221+0000 7f2e227fc700 1 -- 172.21.15.99:0/2138203016 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13741 (secure 0 0 0) 0x7f2e3406bc80 con 0x7f2e34156140 2022-11-27T14:35:17.224 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:35:17.225 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":19,"fsid":"1f55be23-348c-4a1c-92de-37e343596508","created":"2022-11-27T14:34:55.727085+0000","modified":"2022-11-27T14:35:16.510381+0000","last_up_change":"2022-11-27T14:35:06.926207+0000","last_in_change":"2022-11-27T14:34:58.198186+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T14:35:07.033933+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T14:35:14.075015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a3c16e35-9ed8-4726-a5ed-3b34f4dcf80c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":101281},{"type":"v1","addr":"172.21.15.99:6801","nonce":101281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":101281},{"type":"v1","addr":"172.21.15.99:6803","nonce":101281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":101281},{"type":"v1","addr":"172.21.15.99:6807","nonce":101281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":101281},{"type":"v1","addr":"172.21.15.99:6805","nonce":101281}]},"public_addr":"172.21.15.99:6801/101281","cluster_addr":"172.21.15.99:6803/101281","heartbeat_back_addr":"172.21.15.99:6807/101281","heartbeat_front_addr":"172.21.15.99:6805/101281","state":["exists","up"]},{"osd":1,"uuid":"6cb243f2-dcee-4f20-87da-02657be3dce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":101290},{"type":"v1","addr":"172.21.15.99:6809","nonce":101290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":101290},{"type":"v1","addr":"172.21.15.99:6811","nonce":101290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":101290},{"type":"v1","addr":"172.21.15.99:6815","nonce":101290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":101290},{"type":"v1","addr":"172.21.15.99:6813","nonce":101290}]},"public_addr":"172.21.15.99:6809/101290","cluster_addr":"172.21.15.99:6811/101290","heartbeat_back_addr":"172.21.15.99:6815/101290","heartbeat_front_addr":"172.21.15.99:6813/101290","state":["exists","up"]},{"osd":2,"uuid":"6206b34c-8c3d-4337-9f81-27efd1811bd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":101291},{"type":"v1","addr":"172.21.15.99:6817","nonce":101291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":101291},{"type":"v1","addr":"172.21.15.99:6819","nonce":101291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":101291},{"type":"v1","addr":"172.21.15.99:6823","nonce":101291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":101291},{"type":"v1","addr":"172.21.15.99:6821","nonce":101291}]},"public_addr":"172.21.15.99:6817/101291","cluster_addr":"172.21.15.99:6819/101291","heartbeat_back_addr":"172.21.15.99:6823/101291","heartbeat_front_addr":"172.21.15.99:6821/101291","state":["exists","up"]},{"osd":3,"uuid":"b7d6bc6e-1be5-4355-8c30-6ae75fc573e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":101298},{"type":"v1","addr":"172.21.15.99:6825","nonce":101298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":101298},{"type":"v1","addr":"172.21.15.99:6827","nonce":101298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":101298},{"type":"v1","addr":"172.21.15.99:6831","nonce":101298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":101298},{"type":"v1","addr":"172.21.15.99:6829","nonce":101298}]},"public_addr":"172.21.15.99:6825/101298","cluster_addr":"172.21.15.99:6827/101298","heartbeat_back_addr":"172.21.15.99:6831/101298","heartbeat_front_addr":"172.21.15.99:6829/101298","state":["exists","up"]},{"osd":4,"uuid":"983ec37a-caef-4592-81b1-27347e23ee68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":101499},{"type":"v1","addr":"172.21.15.137:6818","nonce":101499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":101499},{"type":"v1","addr":"172.21.15.137:6822","nonce":101499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":101499},{"type":"v1","addr":"172.21.15.137:6830","nonce":101499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":101499},{"type":"v1","addr":"172.21.15.137:6826","nonce":101499}]},"public_addr":"172.21.15.137:6818/101499","cluster_addr":"172.21.15.137:6822/101499","heartbeat_back_addr":"172.21.15.137:6830/101499","heartbeat_front_addr":"172.21.15.137:6826/101499","state":["exists","up"]},{"osd":5,"uuid":"a7625e4c-0f53-4600-be02-c6d0268818bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":101475},{"type":"v1","addr":"172.21.15.137:6801","nonce":101475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":101475},{"type":"v1","addr":"172.21.15.137:6803","nonce":101475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":101475},{"type":"v1","addr":"172.21.15.137:6807","nonce":101475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":101475},{"type":"v1","addr":"172.21.15.137:6805","nonce":101475}]},"public_addr":"172.21.15.137:6801/101475","cluster_addr":"172.21.15.137:6803/101475","heartbeat_back_addr":"172.21.15.137:6807/101475","heartbeat_front_addr":"172.21.15.137:6805/101475","state":["exists","up"]},{"osd":6,"uuid":"c06a48ca-e3d8-45e1-b94f-9a391541ba8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":101476},{"type":"v1","addr":"172.21.15.137:6809","nonce":101476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":101476},{"type":"v1","addr":"172.21.15.137:6811","nonce":101476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":101476},{"type":"v1","addr":"172.21.15.137:6815","nonce":101476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":101476},{"type":"v1","addr":"172.21.15.137:6813","nonce":101476}]},"public_addr":"172.21.15.137:6809/101476","cluster_addr":"172.21.15.137:6811/101476","heartbeat_back_addr":"172.21.15.137:6815/101476","heartbeat_front_addr":"172.21.15.137:6813/101476","state":["exists","up"]},{"osd":7,"uuid":"2302634c-0ed5-4467-ac3c-805b93519b9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6817","nonce":101506},{"type":"v1","addr":"172.21.15.137:6819","nonce":101506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6821","nonce":101506},{"type":"v1","addr":"172.21.15.137:6823","nonce":101506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6829","nonce":101506},{"type":"v1","addr":"172.21.15.137:6831","nonce":101506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6825","nonce":101506},{"type":"v1","addr":"172.21.15.137:6827","nonce":101506}]},"public_addr":"172.21.15.137:6819/101506","cluster_addr":"172.21.15.137:6823/101506","heartbeat_back_addr":"172.21.15.137:6831/101506","heartbeat_front_addr":"172.21.15.137:6827/101506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.371397+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.413296+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.430729+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.359936+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.443571+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.389589+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.404518+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T14:35:03.449713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T14:35:17.227 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f2e14066590 msgr2=0x7f2e14068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.228 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f2e14066590 0x7f2e14068a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009930 tx=0x7f2e30008040 comp rx=0 tx=0).stop 2022-11-27T14:35:17.228 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e34156140 msgr2=0x7f2e340a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.229 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e34156140 0x7f2e340a6a80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00cb60 tx=0x7f2e2c00cf20 comp rx=0 tx=0).stop 2022-11-27T14:35:17.229 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 shutdown_connections 2022-11-27T14:35:17.230 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f2e14066590 0x7f2e14068a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.231 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e34156140 0x7f2e340a6a80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.231 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f2e340bf820 0x7f2e34152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.231 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 --2- 172.21.15.99:0/2138203016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e3414cba0 0x7f2e341eb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.232 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 >> 172.21.15.99:0/2138203016 conn(0x7f2e340a77f0 msgr2=0x7f2e34156570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:17.232 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 shutdown_connections 2022-11-27T14:35:17.233 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.223+0000 7f2e3bdca700 1 -- 172.21.15.99:0/2138203016 wait complete. 2022-11-27T14:35:17.237 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T14:35:07.033933+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T14:35:14.075015+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T14:35:17.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T14:35:17.362 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.360+0000 7f81865c1700 1 Processor -- start 2022-11-27T14:35:17.362 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.360+0000 7f81865c1700 1 -- start start 2022-11-27T14:35:17.370 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f81865c1700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8180069380 0x7f8180069750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.371 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f81865c1700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f8180152a50 0x7f8180152e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.371 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f81865c1700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81800bfa90 con 0x7f81800a4ed0 2022-11-27T14:35:17.371 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f81865c1700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800c1cb0 con 0x7f8180069380 2022-11-27T14:35:17.372 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f81865c1700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800c2230 con 0x7f8180152a50 2022-11-27T14:35:17.372 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8180069380 0x7f8180069750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.373 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817f7fe700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f81800a4ed0 0x7f8180156a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51158/0 (socket says 172.21.15.99:51158) 2022-11-27T14:35:17.373 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817ffff700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8180069380 0x7f8180069750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55184/0 (socket says 172.21.15.99:55184) 2022-11-27T14:35:17.374 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817f7fe700 1 -- 172.21.15.99:0/4287956575 learned_addr learned my addr 172.21.15.99:0/4287956575 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:35:17.374 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f8184b5e700 1 --2- 172.21.15.99:0/4287956575 >> v2:172.21.15.137:3300/0 conn(0x7f8180152a50 0x7f8180152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.375 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817ffff700 1 -- 172.21.15.99:0/4287956575 >> v2:172.21.15.137:3300/0 conn(0x7f8180152a50 msgr2=0x7f8180152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:17.375 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817ffff700 1 --2- 172.21.15.99:0/4287956575 >> v2:172.21.15.137:3300/0 conn(0x7f8180152a50 0x7f8180152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.376 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817ffff700 1 -- 172.21.15.99:0/4287956575 >> v1:172.21.15.99:6789/0 conn(0x7f81800a4ed0 legacy=0x7f8180156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.376 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817ffff700 1 -- 172.21.15.99:0/4287956575 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800c1f50 con 0x7f8180069380 2022-11-27T14:35:17.376 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817effd700 1 -- 172.21.15.99:0/4287956575 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471683346 0 0) 0x7f81800bfa90 con 0x7f81800a4ed0 2022-11-27T14:35:17.376 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f8184b5e700 1 --2- 172.21.15.99:0/4287956575 >> v2:172.21.15.137:3300/0 conn(0x7f8180152a50 0x7f8180152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T14:35:17.377 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.361+0000 7f817ffff700 1 --2- 172.21.15.99:0/4287956575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8180069380 0x7f8180069750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f81800c5a70 tx=0x7f8174009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1f897c707b3f98 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.377 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f817effd700 1 -- 172.21.15.99:0/4287956575 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174011830 con 0x7f8180069380 2022-11-27T14:35:17.377 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f817effd700 1 -- 172.21.15.99:0/4287956575 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174011990 con 0x7f8180069380 2022-11-27T14:35:17.378 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f817effd700 1 -- 172.21.15.99:0/4287956575 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174011d20 con 0x7f8180069380 2022-11-27T14:35:17.378 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f81865c1700 1 -- 172.21.15.99:0/4287956575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8180069380 msgr2=0x7f8180069750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.378 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f81865c1700 1 --2- 172.21.15.99:0/4287956575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8180069380 0x7f8180069750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f81800c5a70 tx=0x7f8174009b20 comp rx=0 tx=0).stop 2022-11-27T14:35:17.378 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f81865c1700 1 -- 172.21.15.99:0/4287956575 shutdown_connections 2022-11-27T14:35:17.379 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f81865c1700 1 --2- 172.21.15.99:0/4287956575 >> v2:172.21.15.137:3300/0 conn(0x7f8180152a50 0x7f8180152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.379 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f81865c1700 1 --2- 172.21.15.99:0/4287956575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8180069380 0x7f8180069750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.379 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f81865c1700 1 -- 172.21.15.99:0/4287956575 >> 172.21.15.99:0/4287956575 conn(0x7f81800a7760 msgr2=0x7f81800a7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:17.379 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f81865c1700 1 -- 172.21.15.99:0/4287956575 shutdown_connections 2022-11-27T14:35:17.380 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.362+0000 7f81865c1700 1 -- 172.21.15.99:0/4287956575 wait complete. 2022-11-27T14:35:17.380 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.363+0000 7f81865c1700 1 Processor -- start 2022-11-27T14:35:17.380 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.363+0000 7f81865c1700 1 -- start start 2022-11-27T14:35:17.380 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f81865c1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180069380 0x7f8180147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.381 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f81865c1700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81800a4ed0 0x7f8180148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.381 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f81865c1700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8180152a50 0x7f81801eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.381 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f81865c1700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180071990 con 0x7f81800a4ed0 2022-11-27T14:35:17.381 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f81865c1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818014dcf0 con 0x7f8180069380 2022-11-27T14:35:17.381 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f81865c1700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8180148840 con 0x7f8180152a50 2022-11-27T14:35:17.382 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f817f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81800a4ed0 0x7f8180148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.382 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f817f7fe700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81800a4ed0 0x7f8180148300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55190/0 (socket says 172.21.15.99:55190) 2022-11-27T14:35:17.382 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f817f7fe700 1 -- 172.21.15.99:0/2409366934 learned_addr learned my addr 172.21.15.99:0/2409366934 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:17.383 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f8184b5e700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8180152a50 0x7f81801eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.383 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.364+0000 7f817ffff700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180069380 0x7f8180147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.383 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817f7fe700 1 -- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8180152a50 msgr2=0x7f81801eb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.384 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817f7fe700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8180152a50 0x7f81801eb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.384 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817f7fe700 1 -- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180069380 msgr2=0x7f8180147dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.385 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817f7fe700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180069380 0x7f8180147dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.385 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817f7fe700 1 -- 172.21.15.99:0/2409366934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81740094a0 con 0x7f81800a4ed0 2022-11-27T14:35:17.385 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817ffff700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180069380 0x7f8180147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T14:35:17.386 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817f7fe700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81800a4ed0 0x7f8180148300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f817000c570 tx=0x7f817000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.386 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817cff9700 1 -- 172.21.15.99:0/2409366934 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170004c80 con 0x7f81800a4ed0 2022-11-27T14:35:17.386 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817cff9700 1 -- 172.21.15.99:0/2409366934 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170004de0 con 0x7f81800a4ed0 2022-11-27T14:35:17.386 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8180071180 con 0x7f81800a4ed0 2022-11-27T14:35:17.387 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f817cff9700 1 -- 172.21.15.99:0/2409366934 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700072f0 con 0x7f81800a4ed0 2022-11-27T14:35:17.387 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.365+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818014df00 con 0x7f81800a4ed0 2022-11-27T14:35:17.387 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.366+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818014e550 con 0x7f81800a4ed0 2022-11-27T14:35:17.387 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.366+0000 7f817cff9700 1 -- 172.21.15.99:0/2409366934 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f817001a020 con 0x7f81800a4ed0 2022-11-27T14:35:17.388 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.366+0000 7f817cff9700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8160066590 0x7f8160068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.388 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.367+0000 7f817cff9700 1 -- 172.21.15.99:0/2409366934 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8170083080 con 0x7f81800a4ed0 2022-11-27T14:35:17.388 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.367+0000 7f817ffff700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8160066590 0x7f8160068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.389 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.367+0000 7f817ffff700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8160066590 0x7f8160068a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81740090b0 tx=0x7f8174008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.371+0000 7f817cff9700 1 -- 172.21.15.99:0/2409366934 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f8170050ae0 con 0x7f81800a4ed0 2022-11-27T14:35:17.553 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.551+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81800733f0 con 0x7f81800a4ed0 2022-11-27T14:35:17.554 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.552+0000 7f817cff9700 1 -- 172.21.15.99:0/2409366934 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8170054100 con 0x7f81800a4ed0 2022-11-27T14:35:17.554 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2022-11-27T14:35:17.556 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8160066590 msgr2=0x7f8160068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.556 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8160066590 0x7f8160068a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81740090b0 tx=0x7f8174008030 comp rx=0 tx=0).stop 2022-11-27T14:35:17.556 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81800a4ed0 msgr2=0x7f8180148300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.556 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81800a4ed0 0x7f8180148300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f817000c570 tx=0x7f817000c880 comp rx=0 tx=0).stop 2022-11-27T14:35:17.557 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 shutdown_connections 2022-11-27T14:35:17.557 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f8160066590 0x7f8160068a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.557 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f8180152a50 0x7f81801eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.557 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180069380 0x7f8180147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.558 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 --2- 172.21.15.99:0/2409366934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81800a4ed0 0x7f8180148300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.558 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 >> 172.21.15.99:0/2409366934 conn(0x7f81800a7760 msgr2=0x7f8180151ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:17.558 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 shutdown_connections 2022-11-27T14:35:17.558 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.554+0000 7f81865c1700 1 -- 172.21.15.99:0/2409366934 wait complete. 2022-11-27T14:35:17.567 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T14:35:17.690 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.688+0000 7f005d7e7700 1 Processor -- start 2022-11-27T14:35:17.690 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.688+0000 7f005d7e7700 1 -- start start 2022-11-27T14:35:17.697 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.689+0000 7f005d7e7700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f005814d0c0 0x7f00580714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.697 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.689+0000 7f005d7e7700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00580a4e50 con 0x7f005806fa80 2022-11-27T14:35:17.698 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.689+0000 7f005d7e7700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00581504b0 con 0x7f00580c3a60 2022-11-27T14:35:17.698 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.689+0000 7f005d7e7700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0058071a80 con 0x7f005814d0c0 2022-11-27T14:35:17.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.689+0000 7f00567fc700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f005806fa80 0x7f00580c24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51170/0 (socket says 172.21.15.99:51170) 2022-11-27T14:35:17.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.689+0000 7f00567fc700 1 -- 172.21.15.99:0/1536731537 learned_addr learned my addr 172.21.15.99:0/1536731537 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:35:17.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.689+0000 7f0056ffd700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f005814d0c0 0x7f00580714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.689+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626414596 0 0) 0x7f00580a4e50 con 0x7f005806fa80 2022-11-27T14:35:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00440035b0 con 0x7f005806fa80 2022-11-27T14:35:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2605000794 0 0) 0x7f00440035b0 con 0x7f005806fa80 2022-11-27T14:35:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f005814d0c0 msgr2=0x7f00580714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f005814d0c0 0x7f00580714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 >> v1:172.21.15.137:6789/0 conn(0x7f00580c3a60 legacy=0x7f00580c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00580a50f0 con 0x7f005806fa80 2022-11-27T14:35:17.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2287561895 0 0) 0x7f0048001670 con 0x7f005806fa80 2022-11-27T14:35:17.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 >> v1:172.21.15.99:6789/0 conn(0x7f005806fa80 legacy=0x7f00580c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f0044003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.702 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f005814d0c0 0x7f0044004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.702 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044004d90 0x7f0044005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.702 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058071a80 con 0x7f0044003a30 2022-11-27T14:35:17.702 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004400bac0 con 0x7f0044004d90 2022-11-27T14:35:17.703 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004400de60 con 0x7f005814d0c0 2022-11-27T14:35:17.703 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0056ffd700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f0044003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.703 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f00567fc700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f005814d0c0 0x7f0044004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.703 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f00577fe700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044004d90 0x7f0044005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0056ffd700 1 -- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f005814d0c0 msgr2=0x7f0044004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:17.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0056ffd700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f005814d0c0 0x7f0044004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0056ffd700 1 -- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044004d90 msgr2=0x7f0044005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:17.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0056ffd700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0044004d90 0x7f0044005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0056ffd700 1 -- 172.21.15.99:0/1536731537 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00440035b0 con 0x7f0044003a30 2022-11-27T14:35:17.705 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0056ffd700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f0044003e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f004c009b60 tx=0x7f0044005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c1c10977cd08b0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.705 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00e6c0 con 0x7f0044003a30 2022-11-27T14:35:17.705 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c00e820 con 0x7f0044003a30 2022-11-27T14:35:17.705 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.690+0000 7f0055ffb700 1 -- 172.21.15.99:0/1536731537 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00eb90 con 0x7f0044003a30 2022-11-27T14:35:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.691+0000 7f005d7e7700 1 -- 172.21.15.99:0/1536731537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 msgr2=0x7f0044003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.691+0000 7f005d7e7700 1 --2- 172.21.15.99:0/1536731537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f0044003e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f004c009b60 tx=0x7f0044005880 comp rx=0 tx=0).stop 2022-11-27T14:35:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.691+0000 7f00577fe700 1 -- 172.21.15.99:0/1536731537 reap_dead start 2022-11-27T14:35:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.691+0000 7f005d7e7700 1 -- 172.21.15.99:0/1536731537 shutdown_connections 2022-11-27T14:35:17.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.691+0000 7f005d7e7700 1 -- 172.21.15.99:0/1536731537 >> 172.21.15.99:0/1536731537 conn(0x7f00580a7760 msgr2=0x7f00580a7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.691+0000 7f005d7e7700 1 -- 172.21.15.99:0/1536731537 shutdown_connections 2022-11-27T14:35:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.691+0000 7f005d7e7700 1 -- 172.21.15.99:0/1536731537 wait complete. 2022-11-27T14:35:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.692+0000 7f005d7e7700 1 Processor -- start 2022-11-27T14:35:17.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.692+0000 7f005d7e7700 1 -- start start 2022-11-27T14:35:17.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f005d7e7700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f005814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f005d7e7700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00581e3ad0 0x7f00580a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f005d7e7700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005814c630 0x7f00581e75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f005d7e7700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00581e70e0 con 0x7f0044003a30 2022-11-27T14:35:17.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f005d7e7700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005814f3c0 con 0x7f005814c630 2022-11-27T14:35:17.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f005d7e7700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005814f6a0 con 0x7f00581e3ad0 2022-11-27T14:35:17.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f0056ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f005814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f0056ffd700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f005814acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55208/0 (socket says 172.21.15.99:55208) 2022-11-27T14:35:17.710 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f0056ffd700 1 -- 172.21.15.99:0/1783474607 learned_addr learned my addr 172.21.15.99:0/1783474607 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:17.710 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f00567fc700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00581e3ad0 0x7f00580a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.710 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f00577fe700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005814c630 0x7f00581e75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.710 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f0056ffd700 1 -- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00581e3ad0 msgr2=0x7f00580a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.710 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f0056ffd700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00581e3ad0 0x7f00580a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.711 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f0056ffd700 1 -- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005814c630 msgr2=0x7f00581e75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.711 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f0056ffd700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005814c630 0x7f00581e75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.711 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f0056ffd700 1 -- 172.21.15.99:0/1783474607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005814f980 con 0x7f0044003a30 2022-11-27T14:35:17.711 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.693+0000 7f00567fc700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00581e3ad0 0x7f00580a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T14:35:17.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.694+0000 7f0056ffd700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f005814acd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f004c000c00 tx=0x7f004c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.694+0000 7f003f7fe700 1 -- 172.21.15.99:0/1783474607 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c021070 con 0x7f0044003a30 2022-11-27T14:35:17.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.694+0000 7f003f7fe700 1 -- 172.21.15.99:0/1783474607 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004c0036a0 con 0x7f0044003a30 2022-11-27T14:35:17.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.694+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00581d97a0 con 0x7f0044003a30 2022-11-27T14:35:17.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.694+0000 7f003f7fe700 1 -- 172.21.15.99:0/1783474607 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004c00e6c0 con 0x7f0044003a30 2022-11-27T14:35:17.713 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.694+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00580a73a0 con 0x7f0044003a30 2022-11-27T14:35:17.713 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.695+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00581e7040 con 0x7f0044003a30 2022-11-27T14:35:17.713 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.695+0000 7f003f7fe700 1 -- 172.21.15.99:0/1783474607 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f004c02e030 con 0x7f0044003a30 2022-11-27T14:35:17.713 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.695+0000 7f003f7fe700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f0038066590 0x7f0038068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:17.714 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.695+0000 7f003f7fe700 1 -- 172.21.15.99:0/1783474607 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f004c0848a0 con 0x7f0044003a30 2022-11-27T14:35:17.714 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.695+0000 7f00567fc700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f0038066590 0x7f0038068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:17.714 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.696+0000 7f00567fc700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f0038066590 0x7f0038068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f005806daf0 tx=0x7f0048009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:17.715 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.700+0000 7f003f7fe700 1 -- 172.21.15.99:0/1783474607 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f004c052300 con 0x7f0044003a30 2022-11-27T14:35:17.879 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.877+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0058069050 con 0x7f0044003a30 2022-11-27T14:35:17.880 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.878+0000 7f003f7fe700 1 -- 172.21.15.99:0/1783474607 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f004c017020 con 0x7f0044003a30 2022-11-27T14:35:17.880 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 8 2022-11-27T14:35:17.883 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.879+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f0038066590 msgr2=0x7f0038068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.883 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.879+0000 7f005d7e7700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f0038066590 0x7f0038068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f005806daf0 tx=0x7f0048009040 comp rx=0 tx=0).stop 2022-11-27T14:35:17.884 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.879+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 msgr2=0x7f005814acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:17.884 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.879+0000 7f005d7e7700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f005814acd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f004c000c00 tx=0x7f004c0047c0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.885 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.880+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 shutdown_connections 2022-11-27T14:35:17.885 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.880+0000 7f005d7e7700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f0038066590 0x7f0038068a40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.885 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.880+0000 7f005d7e7700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005814c630 0x7f00581e75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.886 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.880+0000 7f005d7e7700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0044003a30 0x7f005814acd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.887 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.880+0000 7f005d7e7700 1 --2- 172.21.15.99:0/1783474607 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f00581e3ad0 0x7f00580a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:17.887 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.880+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 >> 172.21.15.99:0/1783474607 conn(0x7f00580a7760 msgr2=0x7f00580c5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:17.888 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.880+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 shutdown_connections 2022-11-27T14:35:17.888 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:17.880+0000 7f005d7e7700 1 -- 172.21.15.99:0/1783474607 wait complete. 2022-11-27T14:35:17.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-27T14:35:18.012 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.011+0000 7f6c7856e700 1 Processor -- start 2022-11-27T14:35:18.013 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.011+0000 7f6c7856e700 1 -- start start 2022-11-27T14:35:18.021 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c7856e700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c70069380 0x7f6c70069750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.021 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c7856e700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f6c70152a50 0x7f6c70152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.021 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c7856e700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c700bfa90 con 0x7f6c700a4ed0 2022-11-27T14:35:18.021 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c7856e700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c700c1cb0 con 0x7f6c70152a50 2022-11-27T14:35:18.022 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c7856e700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c2230 con 0x7f6c70069380 2022-11-27T14:35:18.022 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75b09700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6c700a4ed0 0x7f6c70156a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51192/0 (socket says 172.21.15.99:51192) 2022-11-27T14:35:18.022 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75b09700 1 -- 172.21.15.99:0/476237832 learned_addr learned my addr 172.21.15.99:0/476237832 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:35:18.022 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c7630a700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c70069380 0x7f6c70069750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.022 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487710190 0 0) 0x7f6c700bfa90 con 0x7f6c700a4ed0 2022-11-27T14:35:18.023 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c76b0b700 1 --2- 172.21.15.99:0/476237832 >> v2:172.21.15.137:3300/0 conn(0x7f6c70152a50 0x7f6c70152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.023 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c580035b0 con 0x7f6c700a4ed0 2022-11-27T14:35:18.023 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445554681 0 0) 0x7f6c580035b0 con 0x7f6c700a4ed0 2022-11-27T14:35:18.024 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c70069380 msgr2=0x7f6c70069750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:18.024 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c70069380 0x7f6c70069750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.024 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 >> v2:172.21.15.137:3300/0 conn(0x7f6c70152a50 msgr2=0x7f6c70152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:18.024 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 --2- 172.21.15.99:0/476237832 >> v2:172.21.15.137:3300/0 conn(0x7f6c70152a50 0x7f6c70152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.025 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c700c1f50 con 0x7f6c700a4ed0 2022-11-27T14:35:18.025 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2287561895 0 0) 0x7f6c6c001670 con 0x7f6c700a4ed0 2022-11-27T14:35:18.025 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 >> v1:172.21.15.99:6789/0 conn(0x7f6c700a4ed0 legacy=0x7f6c70156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.025 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c58003a00 0x7f6c58003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.026 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c58004430 0x7f6c580048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.026 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c70069380 0x7f6c58004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.026 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c1cb0 con 0x7f6c58003a00 2022-11-27T14:35:18.026 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700c2230 con 0x7f6c58004430 2022-11-27T14:35:18.026 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5800ba10 con 0x7f6c70069380 2022-11-27T14:35:18.027 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.012+0000 7f6c7630a700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c58003a00 0x7f6c58003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.027 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c75b09700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c58004430 0x7f6c580048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.027 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c76b0b700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c70069380 0x7f6c58004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.027 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c7630a700 1 -- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c70069380 msgr2=0x7f6c58004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:18.028 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c7630a700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c70069380 0x7f6c58004fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.028 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c7630a700 1 -- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c58004430 msgr2=0x7f6c580048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:18.028 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c7630a700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c58004430 0x7f6c580048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.028 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c7630a700 1 -- 172.21.15.99:0/476237832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580035b0 con 0x7f6c58003a00 2022-11-27T14:35:18.028 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c75b09700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c58004430 0x7f6c580048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T14:35:18.029 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c7630a700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c58003a00 0x7f6c58003e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009b60 tx=0x7f6c58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f3d668a134befb server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:18.029 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000e6c0 con 0x7f6c58003a00 2022-11-27T14:35:18.029 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6000e820 con 0x7f6c58003a00 2022-11-27T14:35:18.029 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c75308700 1 -- 172.21.15.99:0/476237832 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000eb90 con 0x7f6c58003a00 2022-11-27T14:35:18.029 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c7856e700 1 -- 172.21.15.99:0/476237832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c58003a00 msgr2=0x7f6c58003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.030 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c7856e700 1 --2- 172.21.15.99:0/476237832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c58003a00 0x7f6c58003e10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009b60 tx=0x7f6c58005880 comp rx=0 tx=0).stop 2022-11-27T14:35:18.030 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.013+0000 7f6c76b0b700 1 -- 172.21.15.99:0/476237832 reap_dead start 2022-11-27T14:35:18.030 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.014+0000 7f6c7856e700 1 -- 172.21.15.99:0/476237832 shutdown_connections 2022-11-27T14:35:18.030 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.014+0000 7f6c7856e700 1 -- 172.21.15.99:0/476237832 >> 172.21.15.99:0/476237832 conn(0x7f6c700a7760 msgr2=0x7f6c700a7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:18.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.014+0000 7f6c7856e700 1 -- 172.21.15.99:0/476237832 shutdown_connections 2022-11-27T14:35:18.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.014+0000 7f6c7856e700 1 -- 172.21.15.99:0/476237832 wait complete. 2022-11-27T14:35:18.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.015+0000 7f6c7856e700 1 Processor -- start 2022-11-27T14:35:18.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.015+0000 7f6c7856e700 1 -- start start 2022-11-27T14:35:18.031 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c7856e700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c58003a00 0x7f6c700703f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.032 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c7856e700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c70151700 0x7f6c70141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.032 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c7856e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7014b500 0x7f6c70142230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.032 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c7856e700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c70142770 con 0x7f6c70151700 2022-11-27T14:35:18.032 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c7856e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c700ab190 con 0x7f6c7014b500 2022-11-27T14:35:18.033 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c7856e700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7006fcf0 con 0x7f6c58003a00 2022-11-27T14:35:18.033 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c75b09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c70151700 0x7f6c70141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.033 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c75b09700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c70151700 0x7f6c70141cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55230/0 (socket says 172.21.15.99:55230) 2022-11-27T14:35:18.033 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c75b09700 1 -- 172.21.15.99:0/4104592014 learned_addr learned my addr 172.21.15.99:0/4104592014 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:18.033 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c76b0b700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7014b500 0x7f6c70142230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.034 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c7630a700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c58003a00 0x7f6c700703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.034 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c75b09700 1 -- 172.21.15.99:0/4104592014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c58003a00 msgr2=0x7f6c700703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.034 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c75b09700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c58003a00 0x7f6c700703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.035 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c75b09700 1 -- 172.21.15.99:0/4104592014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7014b500 msgr2=0x7f6c70142230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.035 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c75b09700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7014b500 0x7f6c70142230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.035 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.016+0000 7f6c75b09700 1 -- 172.21.15.99:0/4104592014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c700ab3a0 con 0x7f6c70151700 2022-11-27T14:35:18.035 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.017+0000 7f6c75b09700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c70151700 0x7f6c70141cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00dae0 tx=0x7f6c6c00dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:18.036 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.017+0000 7f6c5effd700 1 -- 172.21.15.99:0/4104592014 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004a80 con 0x7f6c70151700 2022-11-27T14:35:18.036 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.017+0000 7f6c5effd700 1 -- 172.21.15.99:0/4104592014 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c004be0 con 0x7f6c70151700 2022-11-27T14:35:18.036 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.017+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7014fa30 con 0x7f6c70151700 2022-11-27T14:35:18.036 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.017+0000 7f6c5effd700 1 -- 172.21.15.99:0/4104592014 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0070d0 con 0x7f6c70151700 2022-11-27T14:35:18.036 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.017+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c70070930 con 0x7f6c70151700 2022-11-27T14:35:18.037 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.018+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c700c3e10 con 0x7f6c70151700 2022-11-27T14:35:18.037 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.018+0000 7f6c5effd700 1 -- 172.21.15.99:0/4104592014 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f6c6c007230 con 0x7f6c70151700 2022-11-27T14:35:18.037 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.018+0000 7f6c5effd700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6c500665c0 0x7f6c50068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.037 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.018+0000 7f6c5effd700 1 -- 172.21.15.99:0/4104592014 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6c6c082d60 con 0x7f6c70151700 2022-11-27T14:35:18.038 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.018+0000 7f6c7630a700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6c500665c0 0x7f6c50068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.038 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.019+0000 7f6c7630a700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6c500665c0 0x7f6c50068a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c600090b0 tx=0x7f6c60008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:18.038 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.023+0000 7f6c5effd700 1 -- 172.21.15.99:0/4104592014 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f6c6c0507c0 con 0x7f6c70151700 2022-11-27T14:35:18.204 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.203+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6c7006fe60 con 0x7f6c70151700 2022-11-27T14:35:18.205 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.203+0000 7f6c5effd700 1 -- 172.21.15.99:0/4104592014 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f6c6c019020 con 0x7f6c70151700 2022-11-27T14:35:18.205 INFO:teuthology.orchestra.run.smithi099.stdout: 2022-11-27T14:35:18.205 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2022-11-27T14:35:18.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2022-11-27T14:35:18.206 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2022-11-27T14:35:18.206 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2022-11-27T14:35:18.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2022-11-27T14:35:18.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2022-11-27T14:35:18.207 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 2, 2022-11-27T14:35:18.207 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": "rbd" 2022-11-27T14:35:18.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2022-11-27T14:35:18.208 INFO:teuthology.orchestra.run.smithi099.stdout:] 2022-11-27T14:35:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6c500665c0 msgr2=0x7f6c50068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6c500665c0 0x7f6c50068a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c600090b0 tx=0x7f6c60008030 comp rx=0 tx=0).stop 2022-11-27T14:35:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c70151700 msgr2=0x7f6c70141cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.209 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c70151700 0x7f6c70141cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00dae0 tx=0x7f6c6c00dea0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 shutdown_connections 2022-11-27T14:35:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f6c500665c0 0x7f6c50068a70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7014b500 0x7f6c70142230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.210 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f6c58003a00 0x7f6c700703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 --2- 172.21.15.99:0/4104592014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c70151700 0x7f6c70141cf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 >> 172.21.15.99:0/4104592014 conn(0x7f6c700a7760 msgr2=0x7f6c70152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 shutdown_connections 2022-11-27T14:35:18.211 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.206+0000 7f6c7856e700 1 -- 172.21.15.99:0/4104592014 wait complete. 2022-11-27T14:35:18.218 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-27T14:35:18.219 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-11-27T14:35:18.342 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.340+0000 7f3430cd8700 1 Processor -- start 2022-11-27T14:35:18.342 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.340+0000 7f3430cd8700 1 -- start start 2022-11-27T14:35:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.341+0000 7f3430cd8700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f342c154e00 0x7f342c0c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.341+0000 7f3430cd8700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c0c2540 0x7f342c0c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.341+0000 7f3430cd8700 1 -- --> v1:172.21.15.137:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0c5590 con 0x7f342c06de50 2022-11-27T14:35:18.346 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.341+0000 7f3430cd8700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c0a6350 con 0x7f342c154e00 2022-11-27T14:35:18.347 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.341+0000 7f3430cd8700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0c6a90 con 0x7f342c0c2540 2022-11-27T14:35:18.347 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f3429d9b700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c0c2540 0x7f342c0c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.347 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f342a59c700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f342c154e00 0x7f342c0c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.347 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f3429d9b700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c0c2540 0x7f342c0c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3301/0 says I am v2:172.21.15.99:38056/0 (socket says 172.21.15.99:38056) 2022-11-27T14:35:18.348 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f342a59c700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f342c154e00 0x7f342c0c34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.99:43502/0 (socket says 172.21.15.99:43502) 2022-11-27T14:35:18.348 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f3429d9b700 1 -- 172.21.15.99:0/1293587431 learned_addr learned my addr 172.21.15.99:0/1293587431 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f342ad9d700 1 --1- 172.21.15.99:0/1293587431 >> v1:172.21.15.137:6789/0 conn(0x7f342c06de50 0x7f342c06e2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.137:6789/0 says I am v1:172.21.15.99:45242/0 (socket says 172.21.15.99:45242) 2022-11-27T14:35:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f3429d9b700 1 -- 172.21.15.99:0/1293587431 >> v2:172.21.15.137:3300/0 conn(0x7f342c154e00 msgr2=0x7f342c0c34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f3429d9b700 1 --2- 172.21.15.99:0/1293587431 >> v2:172.21.15.137:3300/0 conn(0x7f342c154e00 0x7f342c0c34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.349 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f3429d9b700 1 -- 172.21.15.99:0/1293587431 >> v1:172.21.15.137:6789/0 conn(0x7f342c06de50 legacy=0x7f342c06e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f3429d9b700 1 -- 172.21.15.99:0/1293587431 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c06f300 con 0x7f342c0c2540 2022-11-27T14:35:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f342a59c700 1 --2- 172.21.15.99:0/1293587431 >> v2:172.21.15.137:3300/0 conn(0x7f342c154e00 0x7f342c0c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:35:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f3429d9b700 1 --2- 172.21.15.99:0/1293587431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c0c2540 0x7f342c0c3a10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f341c0098f0 tx=0x7f341c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e37557a2cbcd5f8 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f342959a700 1 -- 172.21.15.99:0/1293587431 <== mon.2 v2:172.21.15.137:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c011830 con 0x7f342c0c2540 2022-11-27T14:35:18.350 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f342959a700 1 -- 172.21.15.99:0/1293587431 <== mon.2 v2:172.21.15.137:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c011990 con 0x7f342c0c2540 2022-11-27T14:35:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.342+0000 7f342959a700 1 -- 172.21.15.99:0/1293587431 <== mon.2 v2:172.21.15.137:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c011d20 con 0x7f342c0c2540 2022-11-27T14:35:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.343+0000 7f3430cd8700 1 -- 172.21.15.99:0/1293587431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c0c2540 msgr2=0x7f342c0c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.351 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.343+0000 7f3430cd8700 1 --2- 172.21.15.99:0/1293587431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c0c2540 0x7f342c0c3a10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f341c0098f0 tx=0x7f341c009c00 comp rx=0 tx=0).stop 2022-11-27T14:35:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.343+0000 7f3430cd8700 1 -- 172.21.15.99:0/1293587431 shutdown_connections 2022-11-27T14:35:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.343+0000 7f3430cd8700 1 --2- 172.21.15.99:0/1293587431 >> v2:172.21.15.137:3300/0 conn(0x7f342c154e00 0x7f342c0c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.352 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.343+0000 7f3430cd8700 1 --2- 172.21.15.99:0/1293587431 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c0c2540 0x7f342c0c3a10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.343+0000 7f3430cd8700 1 -- 172.21.15.99:0/1293587431 >> 172.21.15.99:0/1293587431 conn(0x7f342c069380 msgr2=0x7f342c069750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:18.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.343+0000 7f3430cd8700 1 -- 172.21.15.99:0/1293587431 shutdown_connections 2022-11-27T14:35:18.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.343+0000 7f3430cd8700 1 -- 172.21.15.99:0/1293587431 wait complete. 2022-11-27T14:35:18.353 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.344+0000 7f3430cd8700 1 Processor -- start 2022-11-27T14:35:18.354 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.344+0000 7f3430cd8700 1 -- start start 2022-11-27T14:35:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f3430cd8700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f342c06de50 0x7f342c06b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f3430cd8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f342c0c2540 0x7f342c1dbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f3430cd8700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c154e00 0x7f342c06bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.357 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f3430cd8700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0aab20 con 0x7f342c06de50 2022-11-27T14:35:18.358 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f3430cd8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c154320 con 0x7f342c0c2540 2022-11-27T14:35:18.358 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f3430cd8700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0aa7d0 con 0x7f342c154e00 2022-11-27T14:35:18.358 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f342c06de50 0x7f342c06b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.358 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342a59c700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f342c06de50 0x7f342c06b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55242/0 (socket says 172.21.15.99:55242) 2022-11-27T14:35:18.359 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342a59c700 1 -- 172.21.15.99:0/2216519007 learned_addr learned my addr 172.21.15.99:0/2216519007 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:18.359 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f3429d9b700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f342c0c2540 0x7f342c1dbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.359 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342ad9d700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c154e00 0x7f342c06bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.359 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342a59c700 1 -- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c154e00 msgr2=0x7f342c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342a59c700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c154e00 0x7f342c06bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342a59c700 1 -- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f342c0c2540 msgr2=0x7f342c1dbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342a59c700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f342c0c2540 0x7f342c1dbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f342a59c700 1 -- 172.21.15.99:0/2216519007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200094a0 con 0x7f342c06de50 2022-11-27T14:35:18.360 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.345+0000 7f3429d9b700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f342c0c2540 0x7f342c1dbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T14:35:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.346+0000 7f342a59c700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f342c06de50 0x7f342c06b610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f342000ebe0 tx=0x7f3420009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.346+0000 7f341affd700 1 -- 172.21.15.99:0/2216519007 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000c910 con 0x7f342c06de50 2022-11-27T14:35:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.346+0000 7f341affd700 1 -- 172.21.15.99:0/2216519007 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342000ca70 con 0x7f342c06de50 2022-11-27T14:35:18.361 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.346+0000 7f341affd700 1 -- 172.21.15.99:0/2216519007 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342000cde0 con 0x7f342c06de50 2022-11-27T14:35:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.346+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c009550 con 0x7f342c06de50 2022-11-27T14:35:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.346+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c0aa4f0 con 0x7f342c06de50 2022-11-27T14:35:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.347+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c0733f0 con 0x7f342c06de50 2022-11-27T14:35:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.348+0000 7f341affd700 1 -- 172.21.15.99:0/2216519007 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f3420019030 con 0x7f342c06de50 2022-11-27T14:35:18.362 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.348+0000 7f341affd700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f3408066590 0x7f3408068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:18.363 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.348+0000 7f341affd700 1 -- 172.21.15.99:0/2216519007 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3420083d60 con 0x7f342c06de50 2022-11-27T14:35:18.363 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.348+0000 7f3429d9b700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f3408066590 0x7f3408068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:18.363 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.349+0000 7f3429d9b700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f3408066590 0x7f3408068a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f341c009c40 tx=0x7f341c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:18.363 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.352+0000 7f341affd700 1 -- 172.21.15.99:0/2216519007 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f3420051750 con 0x7f342c06de50 2022-11-27T14:35:18.534 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:18.533+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f342c153e00 con 0x7f342c06de50 2022-11-27T14:35:19.532 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.531+0000 7f341affd700 1 -- 172.21.15.99:0/2216519007 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3420054d70 con 0x7f342c06de50 2022-11-27T14:35:19.537 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f3408066590 msgr2=0x7f3408068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:19.537 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f3408066590 0x7f3408068a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f341c009c40 tx=0x7f341c008030 comp rx=0 tx=0).stop 2022-11-27T14:35:19.537 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f342c06de50 msgr2=0x7f342c06b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:19.538 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f342c06de50 0x7f342c06b610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f342000ebe0 tx=0x7f3420009ec0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.538 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 shutdown_connections 2022-11-27T14:35:19.538 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f3408066590 0x7f3408068a40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.539 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f342c154e00 0x7f342c06bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.539 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f342c06de50 0x7f342c06b610 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.539 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 --2- 172.21.15.99:0/2216519007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f342c0c2540 0x7f342c1dbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.540 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 >> 172.21.15.99:0/2216519007 conn(0x7f342c069380 msgr2=0x7f342c151570 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:19.540 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.536+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 shutdown_connections 2022-11-27T14:35:19.541 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.537+0000 7f3430cd8700 1 -- 172.21.15.99:0/2216519007 wait complete. 2022-11-27T14:35:19.541 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2022-11-27T14:35:19.553 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2022-11-27T14:35:19.559 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T14:35:19.599 DEBUG:teuthology.orchestra.run.smithi137:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T14:35:19.691 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f718642d700 1 Processor -- start 2022-11-27T14:35:19.692 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f718642d700 1 -- start start 2022-11-27T14:35:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f718642d700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7180069380 0x7f7180069750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.698 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f718642d700 1 --2- >> v2:172.21.15.137:3300/0 conn(0x7f71800a4ed0 0x7f7180156a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f718642d700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71800bfa90 con 0x7f7180152a50 2022-11-27T14:35:19.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f718642d700 1 -- --> v2:172.21.15.137:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800c1cb0 con 0x7f71800a4ed0 2022-11-27T14:35:19.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f718642d700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7180156f90 con 0x7f7180069380 2022-11-27T14:35:19.699 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f71849ca700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7180152a50 0x7f7180152e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51228/0 (socket says 172.21.15.99:51228) 2022-11-27T14:35:19.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.691+0000 7f71849ca700 1 -- 172.21.15.99:0/911372464 learned_addr learned my addr 172.21.15.99:0/911372464 (peer_addr_for_me v1:172.21.15.99:0/0) 2022-11-27T14:35:19.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717ffff700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7180069380 0x7f7180069750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:19.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048617644 0 0) 0x7f71800bfa90 con 0x7f7180152a50 2022-11-27T14:35:19.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717f7fe700 1 --2- 172.21.15.99:0/911372464 >> v2:172.21.15.137:3300/0 conn(0x7f71800a4ed0 0x7f7180156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:19.700 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f716c0035b0 con 0x7f7180152a50 2022-11-27T14:35:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2820424315 0 0) 0x7f716c0035b0 con 0x7f7180152a50 2022-11-27T14:35:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7180069380 msgr2=0x7f7180069750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7180069380 0x7f7180069750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.701 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 >> v2:172.21.15.137:3300/0 conn(0x7f71800a4ed0 msgr2=0x7f7180156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 --2- 172.21.15.99:0/911372464 >> v2:172.21.15.137:3300/0 conn(0x7f71800a4ed0 0x7f7180156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800c2230 con 0x7f7180152a50 2022-11-27T14:35:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2287561895 0 0) 0x7f71780019f0 con 0x7f7180152a50 2022-11-27T14:35:19.702 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 >> v1:172.21.15.99:6789/0 conn(0x7f7180152a50 legacy=0x7f7180152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:19.703 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f716c003a00 0x7f716c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.703 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7180069380 0x7f716c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.703 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f716c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.703 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800c1cb0 con 0x7f716c004d90 2022-11-27T14:35:19.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7180156f90 con 0x7f716c003a00 2022-11-27T14:35:19.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c00ba10 con 0x7f7180069380 2022-11-27T14:35:19.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f71849ca700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f716c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:19.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717ffff700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f716c003a00 0x7f716c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:19.704 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f71849ca700 1 -- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7180069380 msgr2=0x7f716c004850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2022-11-27T14:35:19.705 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f71849ca700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f7180069380 0x7f716c004850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.705 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f71849ca700 1 -- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f716c003a00 msgr2=0x7f716c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T14:35:19.705 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f71849ca700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f716c003a00 0x7f716c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.705 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f71849ca700 1 -- 172.21.15.99:0/911372464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c0035b0 con 0x7f716c004d90 2022-11-27T14:35:19.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.692+0000 7f717ffff700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f716c003a00 0x7f716c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T14:35:19.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f71849ca700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f716c005200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f717800a830 tx=0x7f716c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c09569d62060070 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:19.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178012600 con 0x7f716c004d90 2022-11-27T14:35:19.706 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178012760 con 0x7f716c004d90 2022-11-27T14:35:19.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f717effd700 1 -- 172.21.15.99:0/911372464 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178012b10 con 0x7f716c004d90 2022-11-27T14:35:19.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f718642d700 1 -- 172.21.15.99:0/911372464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 msgr2=0x7f716c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:19.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f718642d700 1 --2- 172.21.15.99:0/911372464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f716c005200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f717800a830 tx=0x7f716c005880 comp rx=0 tx=0).stop 2022-11-27T14:35:19.707 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f71849ca700 1 -- 172.21.15.99:0/911372464 reap_dead start 2022-11-27T14:35:19.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f718642d700 1 -- 172.21.15.99:0/911372464 shutdown_connections 2022-11-27T14:35:19.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f718642d700 1 -- 172.21.15.99:0/911372464 >> 172.21.15.99:0/911372464 conn(0x7f71800a7760 msgr2=0x7f71800a7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T14:35:19.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f718642d700 1 -- 172.21.15.99:0/911372464 shutdown_connections 2022-11-27T14:35:19.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.693+0000 7f718642d700 1 -- 172.21.15.99:0/911372464 wait complete. 2022-11-27T14:35:19.708 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f718642d700 1 Processor -- start 2022-11-27T14:35:19.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f718642d700 1 -- start start 2022-11-27T14:35:19.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f718642d700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f71800a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f718642d700 1 --2- >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f718014d950 0x7f718014df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f718642d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f718014e470 0x7f718006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.709 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f718642d700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718014f150 con 0x7f716c004d90 2022-11-27T14:35:19.710 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f718642d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718014e910 con 0x7f718014e470 2022-11-27T14:35:19.710 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f718642d700 1 -- --> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7180071920 con 0x7f718014d950 2022-11-27T14:35:19.710 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f717ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f71800a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:19.711 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f717ffff700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f71800a6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55262/0 (socket says 172.21.15.99:55262) 2022-11-27T14:35:19.711 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.695+0000 7f717ffff700 1 -- 172.21.15.99:0/1694774078 learned_addr learned my addr 172.21.15.99:0/1694774078 (peer_addr_for_me v2:172.21.15.99:0/0) 2022-11-27T14:35:19.711 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717f7fe700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f718014d950 0x7f718014df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:19.711 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f71849ca700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f718014e470 0x7f718006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:19.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717ffff700 1 -- 172.21.15.99:0/1694774078 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f718014d950 msgr2=0x7f718014df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:19.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717ffff700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f718014d950 0x7f718014df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717ffff700 1 -- 172.21.15.99:0/1694774078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f718014e470 msgr2=0x7f718006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:19.712 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717ffff700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f718014e470 0x7f718006c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T14:35:19.713 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717ffff700 1 -- 172.21.15.99:0/1694774078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71801ec5d0 con 0x7f716c004d90 2022-11-27T14:35:19.713 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717f7fe700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.137:3301/0,v1:172.21.15.137:6790/0] conn(0x7f718014d950 0x7f718014df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T14:35:19.713 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f71849ca700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f718014e470 0x7f718006c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T14:35:19.714 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717ffff700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f71800a6780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7174011d00 tx=0x7f717400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:19.714 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717cff9700 1 -- 172.21.15.99:0/1694774078 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174011ea0 con 0x7f716c004d90 2022-11-27T14:35:19.714 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717cff9700 1 -- 172.21.15.99:0/1694774078 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717400fd40 con 0x7f716c004d90 2022-11-27T14:35:19.714 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f718642d700 1 -- 172.21.15.99:0/1694774078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71801e85a0 con 0x7f716c004d90 2022-11-27T14:35:19.715 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f717cff9700 1 -- 172.21.15.99:0/1694774078 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174004220 con 0x7f716c004d90 2022-11-27T14:35:19.715 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.696+0000 7f718642d700 1 -- 172.21.15.99:0/1694774078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71801e8b60 con 0x7f716c004d90 2022-11-27T14:35:19.715 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.697+0000 7f718642d700 1 -- 172.21.15.99:0/1694774078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71801e8d80 con 0x7f716c004d90 2022-11-27T14:35:19.715 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.697+0000 7f717cff9700 1 -- 172.21.15.99:0/1694774078 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f7174024070 con 0x7f716c004d90 2022-11-27T14:35:19.716 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.698+0000 7f717cff9700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f71600665c0 0x7f7160068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T14:35:19.717 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.698+0000 7f717f7fe700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f71600665c0 0x7f7160068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T14:35:19.717 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.698+0000 7f717cff9700 1 -- 172.21.15.99:0/1694774078 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f7174083a70 con 0x7f716c004d90 2022-11-27T14:35:19.717 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.698+0000 7f717f7fe700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f71600665c0 0x7f7160068a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7180151f60 tx=0x7f7170008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T14:35:19.717 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.702+0000 7f717cff9700 1 -- 172.21.15.99:0/1694774078 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f7174051360 con 0x7f716c004d90 2022-11-27T14:35:19.882 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:19.882+0000 7f718642d700 1 -- 172.21.15.99:0/1694774078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f71800733f0 con 0x7f716c004d90 2022-11-27T14:35:20.543 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:20.542+0000 7f717cff9700 1 -- 172.21.15.99:0/1694774078 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7174054980 con 0x7f716c004d90 2022-11-27T14:35:20.549 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:20.549+0000 7f718642d700 1 -- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f71600665c0 msgr2=0x7f7160068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:20.550 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:20.549+0000 7f718642d700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn(0x7f71600665c0 0x7f7160068a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7180151f60 tx=0x7f7170008040 comp rx=0 tx=0).stop 2022-11-27T14:35:20.550 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:20.549+0000 7f718642d700 1 -- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 msgr2=0x7f71800a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T14:35:20.550 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:20.549+0000 7f718642d700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f716c004d90 0x7f71800a6780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7174011d00 tx=0x7f717400f520 comp rx=0 tx=0).stop 2022-11-27T14:35:20.550 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:20.549+0000 7f718642d700 1 -- 172.21.15.99:0/1694774078 shutdown_connections 2022-11-27T14:35:20.551 INFO:teuthology.orchestra.run.smithi099.stderr:2022-11-27T14:35:20.549+0000 7f718642d700 1 --2- 172.21.15.99:0/1694774078 >> [v2:172.21.15.99:6832/101009,v1:172.21.15.99:6833/101009] conn