2023-06-23T00:50:34.869 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-23T00:50:34.870 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} --owner scheduled_yuriw@teuthology --name yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi --verbose -- /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/orig.config.yaml 2023-06-23T00:50:34.920 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T00:50:35.018 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927 branch: wip-yuri4-testing-2023-06-21-0846-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: yweinste@redhat.com first_in_suite: false job_id: '7311927' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri4-testing-2023-06-21-0846-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-06-21-0846-quincy sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9037 sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: wip-yuri4-testing-2023-06-21-0846-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c targets: smithi005.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOk9iHkStIKdX0e2hhQMYKWsnjcyeUlJFQaprlN9vcrlgJf1zAQW5lMrINiyEFNp3vlF1/GrAcyz+sP9MvS+cw/8LR0T9x0QvVQcfLoXOkTDBQ0cSA0C8V0980X82UkdwUSLQ7Rc4sUFVAUIArN7JKZlBlMljAhMc5Apt1JwYYScC7akB13fw5PgmoG3MalaMfjasoUy2Cxmxew5tWs/8EHRqE1MDShXdgHOpanXR/hvx8JIRhVTOWkfb9X9aHNO1eLiedcd5nQ6Prp3CzU2O3ghJ/W4CGJ7fmB7YSW5b6b0nkgwuyjocXCCva5RIyYNlhk7FvKxMRe5oVZx7VtNG+SzAiSgqY9RjBgy3GK3rnkZXmi+UfAQuFGYXp9l5kzGe3L/NMkm7VC9QpV7ZNU5U/fAvEqVKw3fImJo9wzkoEp1s+PA6e8VVPB7nDRcwB8Dy5XsMImCa4Be17I67BoUCym7fZk7drk9vi0NDk4ELyGcPQmWGgTdGckES8KbgtRys= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVIx9CZ40xo/rF3tqoJjc/54NIwjw2o4uiIp8TEbajXxRciQF9qUi141lvr2Mf8sIm7p605GD9L8u7Mq/N2iSE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-22_15:25:25 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-23T00:50:35.018 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa; will attempt to use it 2023-06-23T00:50:35.019 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks 2023-06-23T00:50:35.020 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-23T00:50:35.021 INFO:teuthology.task.internal:Checking packages... 2023-06-23T00:50:35.050 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '42ddc46e6e6582d8e29a5a982a81886fe028be2c' 2023-06-23T00:50:35.050 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-23T00:50:35.050 INFO:teuthology.packaging:ref: None 2023-06-23T00:50:35.051 INFO:teuthology.packaging:tag: None 2023-06-23T00:50:35.051 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-23T00:50:35.051 INFO:teuthology.packaging:sha1: 42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T00:50:35.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri4-testing-2023-06-21-0846-quincy 2023-06-23T00:50:35.275 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-719.g42ddc46e 2023-06-23T00:50:35.276 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-23T00:50:35.287 INFO:teuthology.task.internal:no buildpackages task found 2023-06-23T00:50:35.287 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-23T00:50:35.297 INFO:teuthology.task.internal:Saving configuration 2023-06-23T00:50:35.322 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-23T00:50:35.335 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T00:50:35.364 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:20.279799', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB2H38ydfqBDEqWmim0ozVVBopMhHVVktXV0DkaTAc6ukdFqxVnDzMQkrh8Ke7336Gh99ty0Ppo4kQqlLQmOVFg='} 2023-06-23T00:50:35.396 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:20.278191', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVIx9CZ40xo/rF3tqoJjc/54NIwjw2o4uiIp8TEbajXxRciQF9qUi141lvr2Mf8sIm7p605GD9L8u7Mq/N2iSE='} 2023-06-23T00:50:35.397 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-23T00:50:35.406 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-06-23T00:50:35.407 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-06-23T00:50:35.407 INFO:teuthology.run_tasks:Running task console_log... 2023-06-23T00:50:35.515 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7febb1e8a040>, signals=[15]) 2023-06-23T00:50:35.517 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-23T00:50:35.526 INFO:teuthology.task.internal:Opening connections... 2023-06-23T00:50:35.527 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T00:50:35.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:50:35.603 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T00:50:35.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:50:35.673 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-23T00:50:35.689 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2023-06-23T00:50:35.709 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2023-06-23T00:50:35.710 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2023-06-23T00:50:35.767 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T00:50:35.767 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.4 (Ootpa)" 2023-06-23T00:50:35.767 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel" 2023-06-23T00:50:35.767 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora" 2023-06-23T00:50:35.767 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.4" 2023-06-23T00:50:35.768 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8" 2023-06-23T00:50:35.768 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-23T00:50:35.768 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2023-06-23T00:50:35.768 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-23T00:50:35.768 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T00:50:35.768 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T00:50:35.768 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T00:50:35.769 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T00:50:35.769 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T00:50:35.769 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-23T00:50:35.769 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T00:50:35.769 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-23T00:50:35.770 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2023-06-23T00:50:35.795 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2023-06-23T00:50:35.814 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2023-06-23T00:50:35.814 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2023-06-23T00:50:35.870 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T00:50:35.870 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="8.4 (Ootpa)" 2023-06-23T00:50:35.870 INFO:teuthology.orchestra.run.smithi152.stdout:ID="rhel" 2023-06-23T00:50:35.870 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="fedora" 2023-06-23T00:50:35.871 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="8.4" 2023-06-23T00:50:35.871 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el8" 2023-06-23T00:50:35.871 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-23T00:50:35.871 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2023-06-23T00:50:35.871 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-23T00:50:35.871 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T00:50:35.871 INFO:teuthology.orchestra.run.smithi152.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T00:50:35.872 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T00:50:35.872 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T00:50:35.872 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T00:50:35.872 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-23T00:50:35.872 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T00:50:35.873 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-23T00:50:35.873 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2023-06-23T00:50:35.894 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-23T00:50:35.911 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-23T00:50:35.919 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-23T00:50:35.920 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2023-06-23T00:50:35.922 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2023-06-23T00:50:35.938 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-23T00:50:35.946 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-23T00:50:35.947 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2023-06-23T00:50:35.979 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2023-06-23T00:50:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-23T00:50:35.995 INFO:teuthology.orchestra.run.smithi152.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-06-23T00:50:35.996 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-23T00:50:36.070 INFO:teuthology.run_tasks:Running task kernel... 2023-06-23T00:50:36.098 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-06-23T00:50:36.099 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-06-23T00:50:36.100 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T00:50:36.100 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T00:50:36.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:50:36.117 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2023-06-23T00:50:36.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:50:36.118 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2023-06-23T00:50:36.175 INFO:teuthology.orchestra.run.smithi152.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:50:36.175 INFO:teuthology.task.kernel:Running kernel on smithi152: 4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:50:36.176 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum install -y kernel 2023-06-23T00:50:36.177 INFO:teuthology.orchestra.run.smithi005.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:50:36.178 INFO:teuthology.task.kernel:Running kernel on smithi005: 4.18.0-305.25.1.el8_4.x86_64 2023-06-23T00:50:36.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2023-06-23T00:50:37.627 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-06-23T00:50:37.628 INFO:teuthology.orchestra.run.smithi005.stdout:Unable to read consumer identity 2023-06-23T00:50:37.668 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-06-23T00:50:37.668 INFO:teuthology.orchestra.run.smithi152.stdout:Unable to read consumer identity 2023-06-23T00:50:41.923 INFO:teuthology.orchestra.run.smithi152.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-06-23T00:50:41.932 INFO:teuthology.orchestra.run.smithi005.stdout:Copr repo for python3-asyncssh owned by ceph 1.1 kB/s | 3.5 kB 00:03 2023-06-23T00:50:44.088 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 16 MB 00:02 2023-06-23T00:50:44.105 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 16 MB 00:02 2023-06-23T00:50:47.359 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 4.1 MB/s | 24 kB 00:00 2023-06-23T00:50:47.556 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-06-23T00:50:48.087 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-06-23T00:50:48.088 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-06-23T00:50:48.088 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-06-23T00:50:48.253 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-06-23T00:50:48.254 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-06-23T00:50:48.254 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-06-23T00:50:48.287 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-06-23T00:50:48.287 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-06-23T00:50:48.287 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-06-23T00:50:48.358 DEBUG:teuthology.orchestra.run.smithi152:> echo no | sudo yum reinstall kernel || true 2023-06-23T00:50:48.450 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-06-23T00:50:48.451 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2023-06-23T00:50:48.451 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-06-23T00:50:48.572 DEBUG:teuthology.orchestra.run.smithi005:> echo no | sudo yum reinstall kernel || true 2023-06-23T00:50:48.707 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-06-23T00:50:48.708 INFO:teuthology.orchestra.run.smithi152.stdout:Unable to read consumer identity 2023-06-23T00:50:48.977 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-06-23T00:50:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:Unable to read consumer identity 2023-06-23T00:50:48.996 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:01 ago on Fri 23 Jun 2023 12:50:47 AM UTC. 2023-06-23T00:50:49.153 INFO:teuthology.orchestra.run.smithi152.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-23T00:50:49.154 INFO:teuthology.orchestra.run.smithi152.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:50:49.154 INFO:teuthology.orchestra.run.smithi152.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:50:49.177 INFO:teuthology.orchestra.run.smithi152.stderr:Error: No packages marked for reinstall. 2023-06-23T00:50:49.225 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum reinstall -y kernel || true 2023-06-23T00:50:49.279 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 12:50:47 AM UTC. 2023-06-23T00:50:49.428 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-23T00:50:49.429 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:50:49.429 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:50:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:Error: No packages marked for reinstall. 2023-06-23T00:50:49.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum reinstall -y kernel || true 2023-06-23T00:50:49.569 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-06-23T00:50:49.569 INFO:teuthology.orchestra.run.smithi152.stdout:Unable to read consumer identity 2023-06-23T00:50:49.849 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 12:50:47 AM UTC. 2023-06-23T00:50:49.887 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-06-23T00:50:49.888 INFO:teuthology.orchestra.run.smithi005.stdout:Unable to read consumer identity 2023-06-23T00:50:49.977 INFO:teuthology.orchestra.run.smithi152.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-23T00:50:49.978 INFO:teuthology.orchestra.run.smithi152.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:50:49.978 INFO:teuthology.orchestra.run.smithi152.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:50:49.990 INFO:teuthology.orchestra.run.smithi152.stderr:Error: No packages marked for reinstall. 2023-06-23T00:50:50.040 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T00:50:50.164 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.165 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.165 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.165 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-23T00:50:50.166 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-23T00:50:50.166 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-06-23T00:50:50.166 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi152.front.sepia.ceph.com, path=None, version=distro) 2023-06-23T00:50:50.166 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T00:50:50.179 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:03 ago on Fri 23 Jun 2023 12:50:47 AM UTC. 2023-06-23T00:50:50.290 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.291 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.291 DEBUG:teuthology.orchestra.run.smithi152:> sudo rpm -qi grub2-tools 2023-06-23T00:50:50.318 INFO:teuthology.orchestra.run.smithi152.stdout:Name : grub2-tools 2023-06-23T00:50:50.318 INFO:teuthology.orchestra.run.smithi152.stdout:Epoch : 1 2023-06-23T00:50:50.318 INFO:teuthology.orchestra.run.smithi152.stdout:Version : 2.02 2023-06-23T00:50:50.318 INFO:teuthology.orchestra.run.smithi152.stdout:Release : 99.el8_4.1 2023-06-23T00:50:50.319 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: x86_64 2023-06-23T00:50:50.319 INFO:teuthology.orchestra.run.smithi152.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-06-23T00:50:50.319 INFO:teuthology.orchestra.run.smithi152.stdout:Group : System Environment/Base 2023-06-23T00:50:50.319 INFO:teuthology.orchestra.run.smithi152.stdout:Size : 9264631 2023-06-23T00:50:50.319 INFO:teuthology.orchestra.run.smithi152.stdout:License : GPLv3+ 2023-06-23T00:50:50.319 INFO:teuthology.orchestra.run.smithi152.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-06-23T00:50:50.320 INFO:teuthology.orchestra.run.smithi152.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-06-23T00:50:50.320 INFO:teuthology.orchestra.run.smithi152.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-06-23T00:50:50.320 INFO:teuthology.orchestra.run.smithi152.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-06-23T00:50:50.320 INFO:teuthology.orchestra.run.smithi152.stdout:Relocations : (not relocatable) 2023-06-23T00:50:50.320 INFO:teuthology.orchestra.run.smithi152.stdout:Packager : Red Hat, Inc. 2023-06-23T00:50:50.320 INFO:teuthology.orchestra.run.smithi152.stdout:Vendor : Red Hat, Inc. 2023-06-23T00:50:50.321 INFO:teuthology.orchestra.run.smithi152.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-23T00:50:50.321 INFO:teuthology.orchestra.run.smithi152.stdout:Summary : Support tools for GRUB. 2023-06-23T00:50:50.321 INFO:teuthology.orchestra.run.smithi152.stdout:Description : 2023-06-23T00:50:50.321 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T00:50:50.321 INFO:teuthology.orchestra.run.smithi152.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-23T00:50:50.321 INFO:teuthology.orchestra.run.smithi152.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-23T00:50:50.321 INFO:teuthology.orchestra.run.smithi152.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-23T00:50:50.321 INFO:teuthology.orchestra.run.smithi152.stdout:hardware devices. 2023-06-23T00:50:50.322 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T00:50:50.322 INFO:teuthology.orchestra.run.smithi152.stdout:This subpackage provides tools for support of all platforms. 2023-06-23T00:50:50.323 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-23T00:50:50.323 INFO:teuthology.task.kernel:Updating grub on smithi152 to boot 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.323 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-23T00:50:50.336 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-06-23T00:50:50.336 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:50:50.336 INFO:teuthology.orchestra.run.smithi005.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-06-23T00:50:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:Error: No packages marked for reinstall. 2023-06-23T00:50:50.411 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T00:50:50.541 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.541 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.541 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.542 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-06-23T00:50:50.542 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-06-23T00:50:50.542 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-06-23T00:50:50.542 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2023-06-23T00:50:50.542 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T00:50:50.673 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.674 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2023-06-23T00:50:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2023-06-23T00:50:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2023-06-23T00:50:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.02 2023-06-23T00:50:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 99.el8_4.1 2023-06-23T00:50:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2023-06-23T00:50:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-06-23T00:50:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:Group : System Environment/Base 2023-06-23T00:50:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 9264631 2023-06-23T00:50:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2023-06-23T00:50:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-06-23T00:50:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-06-23T00:50:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-06-23T00:50:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-06-23T00:50:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:Relocations : (not relocatable) 2023-06-23T00:50:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : Red Hat, Inc. 2023-06-23T00:50:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : Red Hat, Inc. 2023-06-23T00:50:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-23T00:50:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2023-06-23T00:50:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2023-06-23T00:50:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T00:50:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-23T00:50:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-23T00:50:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-23T00:50:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2023-06-23T00:50:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T00:50:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2023-06-23T00:50:50.707 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-23T00:50:50.707 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:50.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-23T00:50:51.697 INFO:teuthology.orchestra.run.smithi152.stderr:Generating grub configuration file ... 2023-06-23T00:50:51.835 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2023-06-23T00:50:53.948 INFO:teuthology.orchestra.run.smithi152.stderr:done 2023-06-23T00:50:53.950 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2023-06-23T00:50:53.964 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.CWIQQ8CFln 2023-06-23T00:50:53.965 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CWIQQ8CFln 2023-06-23T00:50:54.033 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.CWIQQ8CFln 2023-06-23T00:50:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:done 2023-06-23T00:50:54.109 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2023-06-23T00:50:54.123 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.yvCtciHXn9 2023-06-23T00:50:54.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yvCtciHXn9 2023-06-23T00:50:54.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.yvCtciHXn9 2023-06-23T00:50:54.198 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.CWIQQ8CFln is 6KB 2023-06-23T00:50:54.246 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.CWIQQ8CFln 2023-06-23T00:50:54.260 DEBUG:teuthology.orchestra.run.smithi152:> sudo /bin/ls /boot/loader/entries || true 2023-06-23T00:50:54.325 INFO:teuthology.orchestra.run.smithi152.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-06-23T00:50:54.325 INFO:teuthology.orchestra.run.smithi152.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-06-23T00:50:54.325 INFO:teuthology.orchestra.run.smithi152.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-06-23T00:50:54.325 INFO:teuthology.orchestra.run.smithi152.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-06-23T00:50:54.326 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:54.463 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.yvCtciHXn9 is 6KB 2023-06-23T00:50:54.483 DEBUG:teuthology.orchestra.run.smithi152:> sudo shutdown -r now 2023-06-23T00:50:54.512 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.yvCtciHXn9 2023-06-23T00:50:54.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2023-06-23T00:50:54.592 INFO:teuthology.orchestra.run.smithi005.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-06-23T00:50:54.592 INFO:teuthology.orchestra.run.smithi005.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-06-23T00:50:54.593 INFO:teuthology.orchestra.run.smithi005.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-06-23T00:50:54.593 INFO:teuthology.orchestra.run.smithi005.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-06-23T00:50:54.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-06-23T00:50:54.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2023-06-23T00:51:24.516 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-06-23T00:51:24.517 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-06-23T00:51:24.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:51:24.895 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-06-23T00:51:24.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-06-23T00:51:24.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:51:42.832 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2023-06-23T00:51:43.344 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2023-06-23T00:52:12.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-06-23T00:52:12.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:52:13.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-06-23T00:52:13.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:52:31.216 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2023-06-23T00:52:31.461 DEBUG:teuthology.orchestra.run.smithi005:> true 2023-06-23T00:52:31.727 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-06-23T00:52:31.727 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-425.3.1.el8.x86_64"... 2023-06-23T00:52:31.727 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2023-06-23T00:52:31.784 INFO:teuthology.orchestra.run.smithi005.stdout:4.18.0-425.3.1.el8.x86_64 2023-06-23T00:52:31.784 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:52:31.784 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-23T00:52:31.785 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-23T00:52:32.786 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-06-23T00:52:32.786 DEBUG:teuthology.orchestra.run.smithi005:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T00:52:32.899 INFO:teuthology.orchestra.run.smithi005.stdout:ttyS1 2023-06-23T00:52:32.912 DEBUG:teuthology.parallel:result is None 2023-06-23T00:53:01.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-06-23T00:53:01.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T00:53:01.587 DEBUG:teuthology.orchestra.run.smithi152:> true 2023-06-23T00:53:01.830 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-06-23T00:53:01.830 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-425.3.1.el8.x86_64"... 2023-06-23T00:53:01.830 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2023-06-23T00:53:01.846 INFO:teuthology.orchestra.run.smithi152.stdout:4.18.0-425.3.1.el8.x86_64 2023-06-23T00:53:01.846 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-06-23T00:53:01.847 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-23T00:53:01.847 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-23T00:53:02.849 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-06-23T00:53:02.849 DEBUG:teuthology.orchestra.run.smithi152:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T00:53:02.945 INFO:teuthology.orchestra.run.smithi152.stdout:ttyS1 2023-06-23T00:53:02.964 DEBUG:teuthology.parallel:result is None 2023-06-23T00:53:02.964 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-23T00:53:02.973 INFO:teuthology.task.internal:Creating test directory... 2023-06-23T00:53:02.974 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T00:53:02.978 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T00:53:02.996 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-23T00:53:03.010 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-23T00:53:03.024 INFO:teuthology.task.internal:Creating archive directory... 2023-06-23T00:53:03.025 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T00:53:03.039 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T00:53:03.073 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-23T00:53:03.081 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-23T00:53:03.082 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T00:53:03.104 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T00:53:03.145 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:53:03.150 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:53:03.157 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:53:03.163 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T00:53:03.164 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-23T00:53:03.173 INFO:teuthology.task.internal:Configuring sudo... 2023-06-23T00:53:03.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T00:53:03.176 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T00:53:03.226 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-23T00:53:03.236 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-23T00:53:03.236 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T00:53:03.244 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T00:53:03.283 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:53:03.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:53:03.385 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:53:03.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:53:03.509 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T00:53:03.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T00:53:03.575 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:53:03.620 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T00:53:03.658 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:53:03.743 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T00:53:03.779 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T00:53:03.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T00:53:03.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2023-06-23T00:53:03.847 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2023-06-23T00:53:03.911 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:53:03.937 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T00:53:04.381 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-23T00:53:04.389 INFO:teuthology.task.internal:Starting timer... 2023-06-23T00:53:04.389 INFO:teuthology.run_tasks:Running task pcp... 2023-06-23T00:53:04.404 INFO:teuthology.run_tasks:Running task selinux... 2023-06-23T00:53:04.413 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-23T00:53:04.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2023-06-23T00:53:04.477 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: [ OK ] 2023-06-23T00:53:04.478 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2023-06-23T00:53:04.537 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: [ OK ] 2023-06-23T00:53:04.538 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-23T00:53:04.539 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2023-06-23T00:53:04.565 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2023-06-23T00:53:04.565 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2023-06-23T00:53:04.586 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2023-06-23T00:53:04.587 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive'} 2023-06-23T00:53:04.587 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:53:04.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:53:04.631 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T00:53:04.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T00:53:04.659 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-23T00:53:04.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2023-06-23T00:53:04.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2023-06-23T00:53:04.725 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-23T00:53:04.739 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-06-23T00:53:04.746 INFO:teuthology.repo_utils:Fetching main from origin 2023-06-23T00:53:04.888 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-23T00:53:04.918 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-23T00:53:04.919 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi152.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-23T01:01:31.732 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com')] 2023-06-23T01:01:31.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-06-23T01:01:31.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T01:01:31.805 DEBUG:teuthology.orchestra.run.smithi005:> true 2023-06-23T01:01:31.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-06-23T01:01:31.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T01:01:31.941 DEBUG:teuthology.orchestra.run.smithi152:> true 2023-06-23T01:01:31.964 INFO:teuthology.run_tasks:Running task clock... 2023-06-23T01:01:31.981 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-23T01:01:31.982 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T01:01:31.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:01:31.985 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T01:01:31.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:01:32.015 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T01:01:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T01:01:32.031 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T01:01:32.045 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T01:01:32.065 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2023-06-23T01:01:32.076 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2023-06-23T01:01:32.080 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2023-06-23T01:01:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T01:01:32.091 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2023-06-23T01:01:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T01:01:32.105 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T01:01:32.118 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T01:01:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:bash: ntpq: command not found 2023-06-23T01:01:32.176 INFO:teuthology.orchestra.run.smithi005.stdout:210 Number of sources = 4 2023-06-23T01:01:32.177 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:01:32.177 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2023-06-23T01:01:32.177 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:01:32.178 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:01:32.178 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:01:32.178 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:01:32.201 INFO:teuthology.orchestra.run.smithi152.stderr:bash: ntpq: command not found 2023-06-23T01:01:32.203 INFO:teuthology.orchestra.run.smithi152.stdout:210 Number of sources = 4 2023-06-23T01:01:32.203 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:01:32.204 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2023-06-23T01:01:32.204 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:01:32.204 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:01:32.204 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:01:32.204 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:01:32.204 INFO:teuthology.run_tasks:Running task install... 2023-06-23T01:01:32.213 DEBUG:teuthology.task.install:project ceph 2023-06-23T01:01:32.214 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}} 2023-06-23T01:01:32.214 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'} 2023-06-23T01:01:32.215 INFO:teuthology.task.install:Using flavor: default 2023-06-23T01:01:32.221 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-23T01:01:32.222 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-23T01:01:32.222 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-23T01:01:32.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:01:32.225 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'tag': None, 'wait_for_package': False} 2023-06-23T01:01:32.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:01:32.567 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-23T01:01:32.567 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-23T01:01:32.577 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/ 2023-06-23T01:01:32.577 INFO:teuthology.task.install.rpm:Package version is 17.2.6-719.g42ddc46e 2023-06-23T01:01:32.968 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T01:01:32.969 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:01:32.970 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T01:01:32.974 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-06-21-0846-quincy/42ddc46e6e6582d8e29a5a982a81886fe028be2c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T01:01:32.974 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:01:32.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T01:01:32.996 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-23T01:01:32.997 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T01:01:33.005 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-06-23T01:01:33.005 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/42ddc46e6e6582d8e29a5a982a81886fe028be2c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T01:01:33.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T01:01:33.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T01:01:33.137 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T01:01:33.158 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T01:01:33.203 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2023-06-23T01:01:33.203 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2023-06-23T01:01:33.226 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2023-06-23T01:01:33.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2023-06-23T01:01:33.595 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-06-23T01:01:33.643 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-06-23T01:01:33.887 INFO:teuthology.orchestra.run.smithi152.stdout:44 files removed 2023-06-23T01:01:33.938 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-23T01:01:33.967 INFO:teuthology.orchestra.run.smithi005.stdout:44 files removed 2023-06-23T01:01:34.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-06-23T01:01:34.292 INFO:teuthology.orchestra.run.smithi152.stdout:Updating Subscription Management repositories. 2023-06-23T01:01:34.388 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-06-23T01:01:35.610 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 89 kB/s | 78 kB 00:00 2023-06-23T01:01:35.726 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 91 kB/s | 78 kB 00:00 2023-06-23T01:01:36.028 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 24 kB/s | 9.2 kB 00:00 2023-06-23T01:01:36.120 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 26 kB/s | 9.2 kB 00:00 2023-06-23T01:01:36.588 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 2.7 kB/s | 1.4 kB 00:00 2023-06-23T01:01:36.672 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 2.8 kB/s | 1.4 kB 00:00 2023-06-23T01:01:39.116 INFO:teuthology.orchestra.run.smithi005.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-23T01:01:39.124 INFO:teuthology.orchestra.run.smithi152.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-23T01:01:40.732 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 16 MB 00:01 2023-06-23T01:01:44.198 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 3.1 MB/s | 16 MB 00:05 2023-06-23T01:01:44.276 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-06-23T01:01:44.677 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 105 MB/s | 33 MB 00:00 2023-06-23T01:01:47.434 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-06-23T01:01:47.829 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 122 MB/s | 38 MB 00:00 2023-06-23T01:01:50.240 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 111 MB/s | 38 MB 00:00 2023-06-23T01:01:53.385 INFO:teuthology.orchestra.run.smithi152.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 115 MB/s | 33 MB 00:00 2023-06-23T01:01:59.046 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:01:59.046 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:01:59.058 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-23T01:01:59.058 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-23T01:01:59.059 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-23T01:01:59.060 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T01:01:59.062 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T01:01:59.064 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-23T01:01:59.065 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-23T01:01:59.568 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-06-23T01:01:59.583 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================================================== 2023-06-23T01:01:59.583 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2023-06-23T01:01:59.584 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================================================== 2023-06-23T01:01:59.584 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2023-06-23T01:01:59.585 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T01:01:59.585 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-23T01:01:59.585 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-23T01:01:59.585 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-23T01:01:59.585 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-23T01:01:59.585 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-23T01:01:59.585 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-23T01:01:59.586 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-23T01:01:59.586 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-23T01:01:59.586 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-23T01:01:59.586 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:01:59.586 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-23T01:01:59.586 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-23T01:01:59.586 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-23T01:01:59.587 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-23T01:01:59.587 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-23T01:01:59.587 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-23T01:01:59.587 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T01:01:59.587 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T01:01:59.587 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-23T01:01:59.587 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-23T01:01:59.587 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-23T01:01:59.588 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T01:01:59.588 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-23T01:01:59.588 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T01:01:59.588 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-23T01:01:59.588 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-23T01:01:59.588 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-23T01:01:59.588 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-23T01:01:59.589 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-23T01:01:59.589 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:01:59.589 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-23T01:01:59.589 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-23T01:01:59.589 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2023-06-23T01:01:59.589 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:01:59.589 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-23T01:01:59.590 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2023-06-23T01:01:59.590 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T01:01:59.590 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-23T01:01:59.590 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-23T01:01:59.590 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-23T01:01:59.590 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-23T01:01:59.590 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-23T01:01:59.590 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:01:59.591 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-23T01:01:59.591 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-23T01:01:59.591 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T01:01:59.591 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T01:01:59.591 INFO:teuthology.orchestra.run.smithi005.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-23T01:01:59.591 INFO:teuthology.orchestra.run.smithi005.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T01:01:59.591 INFO:teuthology.orchestra.run.smithi005.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T01:01:59.592 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-23T01:01:59.592 INFO:teuthology.orchestra.run.smithi005.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-23T01:01:59.592 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-23T01:01:59.592 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T01:01:59.592 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-23T01:01:59.592 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T01:01:59.592 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-23T01:01:59.592 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-23T01:01:59.593 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-23T01:01:59.593 INFO:teuthology.orchestra.run.smithi005.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T01:01:59.593 INFO:teuthology.orchestra.run.smithi005.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-23T01:01:59.593 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T01:01:59.593 INFO:teuthology.orchestra.run.smithi005.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T01:01:59.593 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T01:01:59.593 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T01:01:59.593 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-23T01:01:59.594 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T01:01:59.594 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T01:01:59.594 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T01:01:59.594 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T01:01:59.594 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T01:01:59.594 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T01:01:59.594 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T01:01:59.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T01:01:59.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T01:01:59.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-23T01:01:59.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-23T01:01:59.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T01:01:59.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T01:01:59.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T01:01:59.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T01:01:59.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T01:01:59.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T01:01:59.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T01:01:59.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T01:01:59.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T01:01:59.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:01:59.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T01:01:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T01:01:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T01:01:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T01:01:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T01:01:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T01:01:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T01:01:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T01:01:59.597 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T01:01:59.598 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T01:01:59.598 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T01:01:59.598 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T01:01:59.598 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T01:01:59.598 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T01:01:59.598 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T01:01:59.598 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T01:01:59.598 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:01:59.599 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T01:01:59.599 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T01:01:59.599 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-23T01:01:59.599 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T01:01:59.599 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T01:01:59.599 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T01:01:59.599 INFO:teuthology.orchestra.run.smithi005.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T01:01:59.600 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T01:01:59.600 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T01:01:59.600 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T01:01:59.600 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T01:01:59.600 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T01:01:59.600 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T01:01:59.600 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-23T01:01:59.600 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T01:01:59.601 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T01:01:59.601 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T01:01:59.601 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:01:59.601 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-06-23T01:01:59.601 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================================================== 2023-06-23T01:01:59.601 INFO:teuthology.orchestra.run.smithi005.stdout:Install 118 Packages 2023-06-23T01:01:59.601 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 2 Packages 2023-06-23T01:01:59.601 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:01:59.602 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 201 M 2023-06-23T01:01:59.602 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2023-06-23T01:02:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 26 kB/s | 6.4 kB 00:00 2023-06-23T01:02:00.621 INFO:teuthology.orchestra.run.smithi005.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 1.5 MB/s | 873 kB 00:00 2023-06-23T01:02:00.686 INFO:teuthology.orchestra.run.smithi005.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 2.4 MB/s | 162 kB 00:00 2023-06-23T01:02:01.311 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:02:01.312 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T01:02:01.322 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-06-23T01:02:01.323 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-06-23T01:02:01.324 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-06-23T01:02:01.325 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T01:02:01.326 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T01:02:01.328 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-06-23T01:02:01.329 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-06-23T01:02:01.589 INFO:teuthology.orchestra.run.smithi005.stdout:(4/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 2.5 MB/s | 2.3 MB 00:00 2023-06-23T01:02:01.816 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-06-23T01:02:01.827 INFO:teuthology.orchestra.run.smithi152.stdout:====================================================================================================================== 2023-06-23T01:02:01.828 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-06-23T01:02:01.828 INFO:teuthology.orchestra.run.smithi152.stdout:====================================================================================================================== 2023-06-23T01:02:01.828 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-06-23T01:02:01.828 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T01:02:01.828 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 6.4 k 2023-06-23T01:02:01.829 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.2 M 2023-06-23T01:02:01.829 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 873 k 2023-06-23T01:02:01.829 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 162 k 2023-06-23T01:02:01.829 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 1.6 M 2023-06-23T01:02:01.829 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 121 k 2023-06-23T01:02:01.829 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 1.7 M 2023-06-23T01:02:01.829 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 7.4 M 2023-06-23T01:02:01.830 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 47 k 2023-06-23T01:02:01.830 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:02:01.830 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 52 M 2023-06-23T01:02:01.830 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 264 k 2023-06-23T01:02:01.830 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 80 k 2023-06-23T01:02:01.830 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 255 k 2023-06-23T01:02:01.830 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 18 k 2023-06-23T01:02:01.830 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-06-23T01:02:01.831 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T01:02:01.831 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T01:02:01.831 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 25 k 2023-06-23T01:02:01.831 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 794 k 2023-06-23T01:02:01.831 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 130 k 2023-06-23T01:02:01.831 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T01:02:01.831 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-06-23T01:02:01.831 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T01:02:01.832 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 190 k 2023-06-23T01:02:01.832 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 371 k 2023-06-23T01:02:01.832 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 357 k 2023-06-23T01:02:01.832 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 107 k 2023-06-23T01:02:01.832 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 91 k 2023-06-23T01:02:01.832 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:02:01.832 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 173 k 2023-06-23T01:02:01.833 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-06-23T01:02:01.833 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2023-06-23T01:02:01.833 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.5 M 2023-06-23T01:02:01.833 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.6 M 2023-06-23T01:02:01.833 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-06-23T01:02:01.833 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T01:02:01.833 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 23 M 2023-06-23T01:02:01.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 23 k 2023-06-23T01:02:01.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 2.3 M 2023-06-23T01:02:01.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 238 k 2023-06-23T01:02:01.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 3.9 M 2023-06-23T01:02:01.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 17 M 2023-06-23T01:02:01.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:17.2.6-719.g42ddc46e.el8 ceph-noarch 14 k 2023-06-23T01:02:01.834 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 24 k 2023-06-23T01:02:01.834 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T01:02:01.835 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T01:02:01.835 INFO:teuthology.orchestra.run.smithi152.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-06-23T01:02:01.835 INFO:teuthology.orchestra.run.smithi152.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T01:02:01.835 INFO:teuthology.orchestra.run.smithi152.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T01:02:01.835 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 174 k 2023-06-23T01:02:01.835 INFO:teuthology.orchestra.run.smithi152.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-06-23T01:02:01.835 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-06-23T01:02:01.835 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T01:02:01.836 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-06-23T01:02:01.836 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T01:02:01.836 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 506 k 2023-06-23T01:02:01.836 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-06-23T01:02:01.836 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 5.5 M 2023-06-23T01:02:01.836 INFO:teuthology.orchestra.run.smithi152.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T01:02:01.836 INFO:teuthology.orchestra.run.smithi152.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-06-23T01:02:01.837 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T01:02:01.837 INFO:teuthology.orchestra.run.smithi152.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T01:02:01.837 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T01:02:01.837 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T01:02:01.837 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-06-23T01:02:01.837 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T01:02:01.837 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T01:02:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T01:02:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T01:02:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T01:02:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T01:02:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T01:02:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T01:02:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T01:02:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 45 k 2023-06-23T01:02:01.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:17.2.6-719.g42ddc46e.el8 ceph 115 k 2023-06-23T01:02:01.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T01:02:01.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T01:02:01.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T01:02:01.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T01:02:01.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T01:02:01.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T01:02:01.839 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T01:02:01.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T01:02:01.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T01:02:01.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:02:01.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T01:02:01.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T01:02:01.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T01:02:01.840 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T01:02:01.841 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T01:02:01.841 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T01:02:01.841 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T01:02:01.841 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T01:02:01.841 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T01:02:01.841 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T01:02:01.841 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T01:02:01.841 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T01:02:01.842 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T01:02:01.842 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T01:02:01.842 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T01:02:01.842 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T01:02:01.842 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T01:02:01.842 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T01:02:01.842 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T01:02:01.842 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-06-23T01:02:01.843 INFO:teuthology.orchestra.run.smithi152.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T01:02:01.843 INFO:teuthology.orchestra.run.smithi152.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T01:02:01.843 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T01:02:01.843 INFO:teuthology.orchestra.run.smithi152.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T01:02:01.843 INFO:teuthology.orchestra.run.smithi152.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T01:02:01.843 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T01:02:01.843 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T01:02:01.844 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T01:02:01.844 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T01:02:01.844 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T01:02:01.844 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-06-23T01:02:01.844 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T01:02:01.844 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T01:02:01.844 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T01:02:01.845 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:02:01.845 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-06-23T01:02:01.845 INFO:teuthology.orchestra.run.smithi152.stdout:====================================================================================================================== 2023-06-23T01:02:01.845 INFO:teuthology.orchestra.run.smithi152.stdout:Install 118 Packages 2023-06-23T01:02:01.845 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 2 Packages 2023-06-23T01:02:01.845 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:02:01.846 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 201 M 2023-06-23T01:02:01.846 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-06-23T01:02:02.125 INFO:teuthology.orchestra.run.smithi005.stdout:(5/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 2.9 MB/s | 1.6 MB 00:00 2023-06-23T01:02:02.756 INFO:teuthology.orchestra.run.smithi152.stdout:(1/120): ceph-17.2.6-719.g42ddc46e.el8.x86_64.r 7.6 kB/s | 6.4 kB 00:00 2023-06-23T01:02:03.314 INFO:teuthology.orchestra.run.smithi005.stdout:(6/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 1.5 MB/s | 5.2 MB 00:03 2023-06-23T01:02:03.544 INFO:teuthology.orchestra.run.smithi005.stdout:(7/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 2.8 MB/s | 3.9 MB 00:01 2023-06-23T01:02:04.104 INFO:teuthology.orchestra.run.smithi152.stdout:(2/120): ceph-fuse-17.2.6-719.g42ddc46e.el8.x86 648 kB/s | 873 kB 00:01 2023-06-23T01:02:04.290 INFO:teuthology.orchestra.run.smithi152.stdout:(3/120): ceph-immutable-object-cache-17.2.6-719 875 kB/s | 162 kB 00:00 2023-06-23T01:02:04.466 INFO:teuthology.orchestra.run.smithi152.stdout:(4/120): ceph-base-17.2.6-719.g42ddc46e.el8.x86 2.0 MB/s | 5.2 MB 00:02 2023-06-23T01:02:04.864 INFO:teuthology.orchestra.run.smithi005.stdout:(8/120): ceph-common-17.2.6-719.g42ddc46e.el8.x 4.6 MB/s | 23 MB 00:05 2023-06-23T01:02:04.925 INFO:teuthology.orchestra.run.smithi152.stdout:(5/120): ceph-mgr-17.2.6-719.g42ddc46e.el8.x86_ 3.4 MB/s | 1.6 MB 00:00 2023-06-23T01:02:04.934 INFO:teuthology.orchestra.run.smithi005.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 349 kB/s | 24 kB 00:00 2023-06-23T01:02:05.902 INFO:teuthology.orchestra.run.smithi152.stdout:(6/120): ceph-mon-17.2.6-719.g42ddc46e.el8.x86_ 4.0 MB/s | 3.9 MB 00:00 2023-06-23T01:02:06.243 INFO:teuthology.orchestra.run.smithi152.stdout:(7/120): ceph-mds-17.2.6-719.g42ddc46e.el8.x86_ 1.2 MB/s | 2.3 MB 00:01 2023-06-23T01:02:08.520 INFO:teuthology.orchestra.run.smithi005.stdout:(10/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 3.4 MB/s | 17 MB 00:04 2023-06-23T01:02:08.656 INFO:teuthology.orchestra.run.smithi005.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 1.8 MB/s | 255 kB 00:00 2023-06-23T01:02:08.722 INFO:teuthology.orchestra.run.smithi005.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 377 kB/s | 25 kB 00:00 2023-06-23T01:02:09.020 INFO:teuthology.orchestra.run.smithi005.stdout:(13/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 2.6 MB/s | 794 kB 00:00 2023-06-23T01:02:09.133 INFO:teuthology.orchestra.run.smithi005.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 1.5 MB/s | 174 kB 00:00 2023-06-23T01:02:09.192 INFO:teuthology.orchestra.run.smithi005.stdout:(15/120): librados-devel-17.2.6-719.g42ddc46e.e 2.2 MB/s | 130 kB 00:00 2023-06-23T01:02:09.385 INFO:teuthology.orchestra.run.smithi005.stdout:(16/120): libradosstriper1-17.2.6-719.g42ddc46e 2.6 MB/s | 506 kB 00:00 2023-06-23T01:02:09.537 INFO:teuthology.orchestra.run.smithi152.stdout:(8/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86_ 4.6 MB/s | 17 MB 00:03 2023-06-23T01:02:09.612 INFO:teuthology.orchestra.run.smithi152.stdout:(9/120): ceph-selinux-17.2.6-719.g42ddc46e.el8. 324 kB/s | 24 kB 00:00 2023-06-23T01:02:10.584 INFO:teuthology.orchestra.run.smithi005.stdout:(17/120): ceph-osd-17.2.6-719.g42ddc46e.el8.x86 2.3 MB/s | 17 MB 00:07 2023-06-23T01:02:10.655 INFO:teuthology.orchestra.run.smithi005.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 613 kB/s | 45 kB 00:00 2023-06-23T01:02:10.735 INFO:teuthology.orchestra.run.smithi005.stdout:(19/120): python3-ceph-common-17.2.6-719.g42ddc 1.4 MB/s | 115 kB 00:00 2023-06-23T01:02:10.821 INFO:teuthology.orchestra.run.smithi005.stdout:(20/120): python3-cephfs-17.2.6-719.g42ddc46e.e 2.2 MB/s | 190 kB 00:00 2023-06-23T01:02:11.001 INFO:teuthology.orchestra.run.smithi005.stdout:(21/120): python3-rados-17.2.6-719.g42ddc46e.el 2.0 MB/s | 371 kB 00:00 2023-06-23T01:02:11.168 INFO:teuthology.orchestra.run.smithi005.stdout:(22/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 2.1 MB/s | 357 kB 00:00 2023-06-23T01:02:11.202 INFO:teuthology.orchestra.run.smithi005.stdout:(23/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 3.0 MB/s | 5.5 MB 00:01 2023-06-23T01:02:11.255 INFO:teuthology.orchestra.run.smithi005.stdout:(24/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 1.2 MB/s | 107 kB 00:00 2023-06-23T01:02:11.273 INFO:teuthology.orchestra.run.smithi005.stdout:(25/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 1.2 MB/s | 91 kB 00:00 2023-06-23T01:02:11.353 INFO:teuthology.orchestra.run.smithi005.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 2.1 MB/s | 173 kB 00:00 2023-06-23T01:02:11.425 INFO:teuthology.orchestra.run.smithi005.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 322 kB/s | 23 kB 00:00 2023-06-23T01:02:11.495 INFO:teuthology.orchestra.run.smithi005.stdout:(28/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 1.7 MB/s | 121 kB 00:00 2023-06-23T01:02:12.070 INFO:teuthology.orchestra.run.smithi005.stdout:(29/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 2.9 MB/s | 1.7 MB 00:00 2023-06-23T01:02:12.763 INFO:teuthology.orchestra.run.smithi005.stdout:(30/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 2.3 MB/s | 3.5 MB 00:01 2023-06-23T01:02:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 1.7 MB/s | 238 kB 00:00 2023-06-23T01:02:12.958 INFO:teuthology.orchestra.run.smithi005.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 772 kB/s | 47 kB 00:00 2023-06-23T01:02:13.019 INFO:teuthology.orchestra.run.smithi005.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 241 kB/s | 14 kB 00:00 2023-06-23T01:02:13.145 INFO:teuthology.orchestra.run.smithi005.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 2.0 MB/s | 264 kB 00:00 2023-06-23T01:02:13.207 INFO:teuthology.orchestra.run.smithi005.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.3 MB/s | 80 kB 00:00 2023-06-23T01:02:13.279 INFO:teuthology.orchestra.run.smithi005.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 250 kB/s | 18 kB 00:00 2023-06-23T01:02:13.295 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:13.297 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:13.601 INFO:teuthology.orchestra.run.smithi005.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 952 kB/s | 306 kB 00:00 2023-06-23T01:02:13.601 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:13.602 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:13.647 INFO:teuthology.orchestra.run.smithi005.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.3 MB/s | 59 kB 00:00 2023-06-23T01:02:13.647 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:13.648 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:13.691 INFO:teuthology.orchestra.run.smithi005.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.6 MB/s | 75 kB 00:00 2023-06-23T01:02:13.817 INFO:teuthology.orchestra.run.smithi005.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2023-06-23T01:02:13.857 INFO:teuthology.orchestra.run.smithi152.stdout:(10/120): ceph-common-17.2.6-719.g42ddc46e.el8. 2.0 MB/s | 23 MB 00:11 2023-06-23T01:02:13.859 INFO:teuthology.orchestra.run.smithi005.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-06-23T01:02:13.948 INFO:teuthology.orchestra.run.smithi005.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-06-23T01:02:13.987 INFO:teuthology.orchestra.run.smithi005.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 833 kB/s | 33 kB 00:00 2023-06-23T01:02:14.007 INFO:teuthology.orchestra.run.smithi152.stdout:(11/120): cephfs-mirror-17.2.6-719.g42ddc46e.el 1.7 MB/s | 255 kB 00:00 2023-06-23T01:02:14.028 INFO:teuthology.orchestra.run.smithi005.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 399 kB/s | 16 kB 00:00 2023-06-23T01:02:14.079 INFO:teuthology.orchestra.run.smithi005.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 3.3 MB/s | 173 kB 00:00 2023-06-23T01:02:14.082 INFO:teuthology.orchestra.run.smithi152.stdout:(12/120): libcephfs-devel-17.2.6-719.g42ddc46e. 329 kB/s | 25 kB 00:00 2023-06-23T01:02:14.211 INFO:teuthology.orchestra.run.smithi005.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.9 MB/s | 384 kB 00:00 2023-06-23T01:02:14.253 INFO:teuthology.orchestra.run.smithi005.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-06-23T01:02:14.294 INFO:teuthology.orchestra.run.smithi005.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 278 kB/s | 11 kB 00:00 2023-06-23T01:02:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 408 kB/s | 18 kB 00:00 2023-06-23T01:02:14.400 INFO:teuthology.orchestra.run.smithi152.stdout:(13/120): libcephfs2-17.2.6-719.g42ddc46e.el8.x 2.4 MB/s | 794 kB 00:00 2023-06-23T01:02:14.416 INFO:teuthology.orchestra.run.smithi005.stdout:(50/120): ceph-mgr-diskprediction-local-17.2.6- 3.2 MB/s | 7.4 MB 00:02 2023-06-23T01:02:14.474 INFO:teuthology.orchestra.run.smithi152.stdout:(14/120): libcephsqlite-17.2.6-719.g42ddc46e.el 2.3 MB/s | 174 kB 00:00 2023-06-23T01:02:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:(15/120): librados-devel-17.2.6-719.g42ddc46e.e 1.1 MB/s | 130 kB 00:00 2023-06-23T01:02:14.778 INFO:teuthology.orchestra.run.smithi152.stdout:(16/120): libradosstriper1-17.2.6-719.g42ddc46e 2.6 MB/s | 506 kB 00:00 2023-06-23T01:02:14.965 INFO:teuthology.orchestra.run.smithi005.stdout:(51/120): ceph-test-17.2.6-719.g42ddc46e.el8.x8 5.1 MB/s | 52 MB 00:10 2023-06-23T01:02:14.966 INFO:teuthology.orchestra.run.smithi005.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 88 kB/s | 49 kB 00:00 2023-06-23T01:02:14.973 INFO:teuthology.orchestra.run.smithi005.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 1.8 MB/s | 1.2 MB 00:00 2023-06-23T01:02:15.051 INFO:teuthology.orchestra.run.smithi005.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 707 kB/s | 60 kB 00:00 2023-06-23T01:02:15.095 INFO:teuthology.orchestra.run.smithi005.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 361 kB/s | 16 kB 00:00 2023-06-23T01:02:15.101 INFO:teuthology.orchestra.run.smithi005.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-06-23T01:02:15.186 INFO:teuthology.orchestra.run.smithi005.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 369 kB/s | 34 kB 00:00 2023-06-23T01:02:15.212 INFO:teuthology.orchestra.run.smithi005.stdout:(58/120): python3-more-itertools-7.2.0-3.el8.no 238 kB/s | 59 kB 00:00 2023-06-23T01:02:15.305 INFO:teuthology.orchestra.run.smithi005.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 967 kB/s | 196 kB 00:00 2023-06-23T01:02:15.305 INFO:teuthology.orchestra.run.smithi005.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 201 kB/s | 19 kB 00:00 2023-06-23T01:02:15.355 INFO:teuthology.orchestra.run.smithi005.stdout:(61/120): python3-rsa-4.8-1.el8.noarch.rpm 365 kB/s | 61 kB 00:00 2023-06-23T01:02:15.356 INFO:teuthology.orchestra.run.smithi005.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 469 kB/s | 24 kB 00:00 2023-06-23T01:02:15.429 INFO:teuthology.orchestra.run.smithi005.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 372 kB/s | 27 kB 00:00 2023-06-23T01:02:15.503 INFO:teuthology.orchestra.run.smithi005.stdout:(64/120): python3-tempora-1.14.1-5.el8.noarch.r 148 kB/s | 29 kB 00:00 2023-06-23T01:02:15.505 INFO:teuthology.orchestra.run.smithi005.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 1.7 MB/s | 255 kB 00:00 2023-06-23T01:02:15.555 INFO:teuthology.orchestra.run.smithi005.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-06-23T01:02:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:(67/120): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-06-23T01:02:15.604 INFO:teuthology.orchestra.run.smithi005.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 466 kB/s | 23 kB 00:00 2023-06-23T01:02:15.605 INFO:teuthology.orchestra.run.smithi005.stdout:(69/120): python3-websocket-client-0.56.0-5.el8 602 kB/s | 61 kB 00:00 2023-06-23T01:02:15.609 INFO:teuthology.orchestra.run.smithi005.stdout:(70/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 15 MB/s | 70 kB 00:00 2023-06-23T01:02:15.656 INFO:teuthology.orchestra.run.smithi005.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 950 kB/s | 44 kB 00:00 2023-06-23T01:02:15.673 INFO:teuthology.orchestra.run.smithi005.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.5 MB/s | 27 kB 00:00 2023-06-23T01:02:15.695 INFO:teuthology.orchestra.run.smithi005.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 7.5 MB/s | 161 kB 00:00 2023-06-23T01:02:15.717 INFO:teuthology.orchestra.run.smithi005.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 14 MB/s | 320 kB 00:00 2023-06-23T01:02:15.742 INFO:teuthology.orchestra.run.smithi005.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 18 MB/s | 457 kB 00:00 2023-06-23T01:02:15.838 INFO:teuthology.orchestra.run.smithi005.stdout:(76/120): python3-babel-2.5.1-5.el8.noarch.rpm 50 MB/s | 4.8 MB 00:00 2023-06-23T01:02:15.958 INFO:teuthology.orchestra.run.smithi005.stdout:(77/120): python3-scipy-1.0.0-20.module+el8.1.0 41 MB/s | 15 MB 00:00 2023-06-23T01:02:15.959 INFO:teuthology.orchestra.run.smithi005.stdout:(78/120): python3-markupsafe-0.23-19.el8.x86_64 322 kB/s | 39 kB 00:00 2023-06-23T01:02:15.977 INFO:teuthology.orchestra.run.smithi005.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 14 MB/s | 259 kB 00:00 2023-06-23T01:02:15.984 INFO:teuthology.orchestra.run.smithi005.stdout:(80/120): bison-3.0.4-10.el8.x86_64.rpm 26 MB/s | 688 kB 00:00 2023-06-23T01:02:15.997 INFO:teuthology.orchestra.run.smithi005.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 11 MB/s | 225 kB 00:00 2023-06-23T01:02:16.003 INFO:teuthology.orchestra.run.smithi005.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 6.5 MB/s | 126 kB 00:00 2023-06-23T01:02:16.022 INFO:teuthology.orchestra.run.smithi005.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 9.9 MB/s | 187 kB 00:00 2023-06-23T01:02:16.046 INFO:teuthology.orchestra.run.smithi005.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 22 MB/s | 538 kB 00:00 2023-06-23T01:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout:(85/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 52 MB/s | 3.7 MB 00:00 2023-06-23T01:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 11 MB/s | 302 kB 00:00 2023-06-23T01:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 10 MB/s | 157 kB 00:00 2023-06-23T01:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 5.7 MB/s | 110 kB 00:00 2023-06-23T01:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout:(89/120): librdkafka-0.11.4-1.el8.x86_64.rpm 21 MB/s | 353 kB 00:00 2023-06-23T01:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.2 MB/s | 20 kB 00:00 2023-06-23T01:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout:(91/120): libtool-2.4.6-25.el8.x86_64.rpm 32 MB/s | 709 kB 00:00 2023-06-23T01:02:16.124 INFO:teuthology.orchestra.run.smithi005.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.4 MB/s | 54 kB 00:00 2023-06-23T01:02:16.212 INFO:teuthology.orchestra.run.smithi005.stdout:(93/120): openblas-0.3.12-1.el8.x86_64.rpm 54 MB/s | 4.6 MB 00:00 2023-06-23T01:02:16.226 INFO:teuthology.orchestra.run.smithi005.stdout:(94/120): openblas-threads-0.3.12-1.el8.x86_64. 46 MB/s | 4.7 MB 00:00 2023-06-23T01:02:16.236 INFO:teuthology.orchestra.run.smithi005.stdout:(95/120): automake-1.16.1-7.el8.noarch.rpm 27 MB/s | 713 kB 00:00 2023-06-23T01:02:16.244 INFO:teuthology.orchestra.run.smithi005.stdout:(96/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.1 MB/s | 19 kB 00:00 2023-06-23T01:02:16.253 INFO:teuthology.orchestra.run.smithi005.stdout:(97/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.4 MB/s | 43 kB 00:00 2023-06-23T01:02:16.262 INFO:teuthology.orchestra.run.smithi005.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.3 MB/s | 155 kB 00:00 2023-06-23T01:02:16.270 INFO:teuthology.orchestra.run.smithi005.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 12 MB/s | 220 kB 00:00 2023-06-23T01:02:16.278 INFO:teuthology.orchestra.run.smithi005.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.9 MB/s | 48 kB 00:00 2023-06-23T01:02:16.290 INFO:teuthology.orchestra.run.smithi005.stdout:(101/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 12 MB/s | 229 kB 00:00 2023-06-23T01:02:16.293 INFO:teuthology.orchestra.run.smithi005.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 2.8 MB/s | 43 kB 00:00 2023-06-23T01:02:16.309 INFO:teuthology.orchestra.run.smithi005.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.1 MB/s | 18 kB 00:00 2023-06-23T01:02:16.335 INFO:teuthology.orchestra.run.smithi005.stdout:(104/120): openssl-devel-1.1.1g-15.el8_3.x86_64 51 MB/s | 2.3 MB 00:00 2023-06-23T01:02:16.338 INFO:teuthology.orchestra.run.smithi005.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 21 MB/s | 605 kB 00:00 2023-06-23T01:02:16.351 INFO:teuthology.orchestra.run.smithi005.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 5.3 MB/s | 83 kB 00:00 2023-06-23T01:02:16.356 INFO:teuthology.orchestra.run.smithi005.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 3.3 MB/s | 59 kB 00:00 2023-06-23T01:02:16.368 INFO:teuthology.orchestra.run.smithi005.stdout:(108/120): libquadmath-8.4.1-1.el8.x86_64.rpm 9.8 MB/s | 169 kB 00:00 2023-06-23T01:02:16.372 INFO:teuthology.orchestra.run.smithi005.stdout:(109/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.2 MB/s | 38 kB 00:00 2023-06-23T01:02:16.384 INFO:teuthology.orchestra.run.smithi005.stdout:(110/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.3 MB/s | 86 kB 00:00 2023-06-23T01:02:16.388 INFO:teuthology.orchestra.run.smithi005.stdout:(111/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.5 MB/s | 58 kB 00:00 2023-06-23T01:02:16.398 INFO:teuthology.orchestra.run.smithi005.stdout:(112/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.2 MB/s | 47 kB 00:00 2023-06-23T01:02:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:(113/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 24 MB/s | 559 kB 00:00 2023-06-23T01:02:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:(114/120): libgfortran-8.4.1-1.el8.x86_64.rpm 28 MB/s | 639 kB 00:00 2023-06-23T01:02:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 3.8 MB/s | 70 kB 00:00 2023-06-23T01:02:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:(116/120): libselinux-devel-2.9-5.el8.x86_64.rp 8.2 MB/s | 200 kB 00:00 2023-06-23T01:02:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:(117/120): libuuid-devel-2.32.1-27.el8.x86_64.r 3.9 MB/s | 97 kB 00:00 2023-06-23T01:02:17.059 INFO:teuthology.orchestra.run.smithi152.stdout:(17/120): ceph-radosgw-17.2.6-719.g42ddc46e.el8 1.5 MB/s | 17 MB 00:10 2023-06-23T01:02:17.133 INFO:teuthology.orchestra.run.smithi152.stdout:(18/120): python3-ceph-argparse-17.2.6-719.g42d 602 kB/s | 45 kB 00:00 2023-06-23T01:02:17.210 INFO:teuthology.orchestra.run.smithi152.stdout:(19/120): librgw2-17.2.6-719.g42ddc46e.el8.x86_ 2.3 MB/s | 5.5 MB 00:02 2023-06-23T01:02:17.212 INFO:teuthology.orchestra.run.smithi152.stdout:(20/120): python3-ceph-common-17.2.6-719.g42ddc 1.4 MB/s | 115 kB 00:00 2023-06-23T01:02:17.306 INFO:teuthology.orchestra.run.smithi152.stdout:(21/120): python3-cephfs-17.2.6-719.g42ddc46e.e 2.0 MB/s | 190 kB 00:00 2023-06-23T01:02:17.428 INFO:teuthology.orchestra.run.smithi005.stdout:(118/120): thrift-0.13.0-2.el8.x86_64.rpm 970 kB/s | 1.7 MB 00:01 2023-06-23T01:02:17.452 INFO:teuthology.orchestra.run.smithi152.stdout:(22/120): python3-rados-17.2.6-719.g42ddc46e.el 1.5 MB/s | 371 kB 00:00 2023-06-23T01:02:17.612 INFO:teuthology.orchestra.run.smithi005.stdout:(119/120): librados2-17.2.6-719.g42ddc46e.el8.x 3.0 MB/s | 3.5 MB 00:01 2023-06-23T01:02:17.837 INFO:teuthology.orchestra.run.smithi005.stdout:(120/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 2.6 MB/s | 3.6 MB 00:01 2023-06-23T01:02:17.841 INFO:teuthology.orchestra.run.smithi152.stdout:(23/120): python3-rbd-17.2.6-719.g42ddc46e.el8. 666 kB/s | 357 kB 00:00 2023-06-23T01:02:17.843 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:02:17.843 INFO:teuthology.orchestra.run.smithi005.stdout:Total 11 MB/s | 201 MB 00:18 2023-06-23T01:02:17.864 INFO:teuthology.orchestra.run.smithi152.stdout:(24/120): python3-rgw-17.2.6-719.g42ddc46e.el8. 258 kB/s | 107 kB 00:00 2023-06-23T01:02:17.895 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2023-06-23T01:02:17.918 INFO:teuthology.orchestra.run.smithi152.stdout:(25/120): rbd-fuse-17.2.6-719.g42ddc46e.el8.x86 1.2 MB/s | 91 kB 00:00 2023-06-23T01:02:17.980 INFO:teuthology.orchestra.run.smithi152.stdout:(26/120): rbd-nbd-17.2.6-719.g42ddc46e.el8.x86_ 2.7 MB/s | 173 kB 00:00 2023-06-23T01:02:18.019 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2023-06-23T01:02:18.019 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2023-06-23T01:02:18.104 INFO:teuthology.orchestra.run.smithi152.stdout:(27/120): ceph-grafana-dashboards-17.2.6-719.g4 186 kB/s | 23 kB 00:00 2023-06-23T01:02:18.166 INFO:teuthology.orchestra.run.smithi152.stdout:(28/120): ceph-mgr-cephadm-17.2.6-719.g42ddc46e 1.9 MB/s | 121 kB 00:00 2023-06-23T01:02:19.711 INFO:teuthology.orchestra.run.smithi152.stdout:(29/120): ceph-mgr-dashboard-17.2.6-719.g42ddc4 1.1 MB/s | 1.7 MB 00:01 2023-06-23T01:02:19.750 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2023-06-23T01:02:19.754 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2023-06-23T01:02:20.298 INFO:teuthology.orchestra.run.smithi152.stdout:(30/120): rbd-mirror-17.2.6-719.g42ddc46e.el8.x 1.4 MB/s | 3.5 MB 00:02 2023-06-23T01:02:20.427 INFO:teuthology.orchestra.run.smithi152.stdout:(31/120): ceph-mgr-modules-core-17.2.6-719.g42d 1.8 MB/s | 238 kB 00:00 2023-06-23T01:02:20.490 INFO:teuthology.orchestra.run.smithi152.stdout:(32/120): ceph-mgr-rook-17.2.6-719.g42ddc46e.el 756 kB/s | 47 kB 00:00 2023-06-23T01:02:20.552 INFO:teuthology.orchestra.run.smithi152.stdout:(33/120): ceph-prometheus-alerts-17.2.6-719.g42 234 kB/s | 14 kB 00:00 2023-06-23T01:02:20.684 INFO:teuthology.orchestra.run.smithi152.stdout:(34/120): ceph-volume-17.2.6-719.g42ddc46e.el8. 2.0 MB/s | 264 kB 00:00 2023-06-23T01:02:20.749 INFO:teuthology.orchestra.run.smithi152.stdout:(35/120): cephadm-17.2.6-719.g42ddc46e.el8.noar 1.2 MB/s | 80 kB 00:00 2023-06-23T01:02:20.812 INFO:teuthology.orchestra.run.smithi152.stdout:(36/120): cephfs-top-17.2.6-719.g42ddc46e.el8.n 285 kB/s | 18 kB 00:00 2023-06-23T01:02:20.829 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:20.831 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:20.941 INFO:teuthology.orchestra.run.smithi152.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 2.3 MB/s | 306 kB 00:00 2023-06-23T01:02:20.943 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:20.944 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:20.956 INFO:teuthology.orchestra.run.smithi152.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 3.8 MB/s | 59 kB 00:00 2023-06-23T01:02:20.957 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:20.958 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Status code: 404 for http://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm (IP: 8.43.85.72) 2023-06-23T01:02:20.971 INFO:teuthology.orchestra.run.smithi152.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 5.1 MB/s | 75 kB 00:00 2023-06-23T01:02:20.972 INFO:teuthology.orchestra.run.smithi152.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm (IP: 8.43.85.72) 2023-06-23T01:02:21.000 INFO:teuthology.orchestra.run.smithi152.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 16 MB/s | 476 kB 00:00 2023-06-23T01:02:21.012 INFO:teuthology.orchestra.run.smithi152.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 3.6 MB/s | 44 kB 00:00 2023-06-23T01:02:21.026 INFO:teuthology.orchestra.run.smithi152.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 13 MB/s | 185 kB 00:00 2023-06-23T01:02:21.038 INFO:teuthology.orchestra.run.smithi152.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 2.8 MB/s | 33 kB 00:00 2023-06-23T01:02:21.050 INFO:teuthology.orchestra.run.smithi152.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 1.4 MB/s | 16 kB 00:00 2023-06-23T01:02:21.064 INFO:teuthology.orchestra.run.smithi152.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 12 MB/s | 173 kB 00:00 2023-06-23T01:02:21.081 INFO:teuthology.orchestra.run.smithi152.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 23 MB/s | 384 kB 00:00 2023-06-23T01:02:21.093 INFO:teuthology.orchestra.run.smithi152.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 6.3 MB/s | 81 kB 00:00 2023-06-23T01:02:21.106 INFO:teuthology.orchestra.run.smithi152.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 922 kB/s | 11 kB 00:00 2023-06-23T01:02:21.118 INFO:teuthology.orchestra.run.smithi152.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 1.5 MB/s | 18 kB 00:00 2023-06-23T01:02:21.150 INFO:teuthology.orchestra.run.smithi152.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 37 MB/s | 1.2 MB 00:00 2023-06-23T01:02:21.162 INFO:teuthology.orchestra.run.smithi152.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 4.0 MB/s | 49 kB 00:00 2023-06-23T01:02:21.174 INFO:teuthology.orchestra.run.smithi152.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 4.7 MB/s | 59 kB 00:00 2023-06-23T01:02:21.186 INFO:teuthology.orchestra.run.smithi152.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 4.9 MB/s | 60 kB 00:00 2023-06-23T01:02:21.202 INFO:teuthology.orchestra.run.smithi152.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 18 MB/s | 283 kB 00:00 2023-06-23T01:02:21.214 INFO:teuthology.orchestra.run.smithi152.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 1.3 MB/s | 16 kB 00:00 2023-06-23T01:02:21.247 INFO:teuthology.orchestra.run.smithi152.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-06-23T01:02:21.262 INFO:teuthology.orchestra.run.smithi152.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 14 MB/s | 196 kB 00:00 2023-06-23T01:02:21.274 INFO:teuthology.orchestra.run.smithi152.stdout:(58/120): python3-rsa-4.8-1.el8.noarch.rpm 5.0 MB/s | 61 kB 00:00 2023-06-23T01:02:21.285 INFO:teuthology.orchestra.run.smithi152.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 1.6 MB/s | 19 kB 00:00 2023-06-23T01:02:21.297 INFO:teuthology.orchestra.run.smithi152.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 2.0 MB/s | 24 kB 00:00 2023-06-23T01:02:21.309 INFO:teuthology.orchestra.run.smithi152.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 2.4 MB/s | 29 kB 00:00 2023-06-23T01:02:21.321 INFO:teuthology.orchestra.run.smithi152.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 2.3 MB/s | 27 kB 00:00 2023-06-23T01:02:21.336 INFO:teuthology.orchestra.run.smithi152.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 17 MB/s | 255 kB 00:00 2023-06-23T01:02:21.351 INFO:teuthology.orchestra.run.smithi152.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 17 MB/s | 251 kB 00:00 2023-06-23T01:02:21.363 INFO:teuthology.orchestra.run.smithi152.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 4.9 MB/s | 61 kB 00:00 2023-06-23T01:02:21.376 INFO:teuthology.orchestra.run.smithi152.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 6.7 MB/s | 85 kB 00:00 2023-06-23T01:02:21.388 INFO:teuthology.orchestra.run.smithi152.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 1.9 MB/s | 23 kB 00:00 2023-06-23T01:02:21.428 INFO:teuthology.orchestra.run.smithi152.stdout:(68/120): thrift-0.13.0-2.el8.x86_64.rpm 44 MB/s | 1.7 MB 00:00 2023-06-23T01:02:21.432 INFO:teuthology.orchestra.run.smithi152.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 18 MB/s | 70 kB 00:00 2023-06-23T01:02:21.461 INFO:teuthology.orchestra.run.smithi152.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 634 kB/s | 19 kB 00:00 2023-06-23T01:02:21.476 INFO:teuthology.orchestra.run.smithi152.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.9 MB/s | 43 kB 00:00 2023-06-23T01:02:21.490 INFO:teuthology.orchestra.run.smithi152.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 11 MB/s | 155 kB 00:00 2023-06-23T01:02:21.505 INFO:teuthology.orchestra.run.smithi152.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 15 MB/s | 220 kB 00:00 2023-06-23T01:02:21.520 INFO:teuthology.orchestra.run.smithi152.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.1 MB/s | 48 kB 00:00 2023-06-23T01:02:21.536 INFO:teuthology.orchestra.run.smithi152.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 14 MB/s | 229 kB 00:00 2023-06-23T01:02:21.551 INFO:teuthology.orchestra.run.smithi152.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 2.9 MB/s | 43 kB 00:00 2023-06-23T01:02:21.590 INFO:teuthology.orchestra.run.smithi152.stdout:(77/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 60 MB/s | 2.3 MB 00:00 2023-06-23T01:02:21.604 INFO:teuthology.orchestra.run.smithi152.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.3 MB/s | 18 kB 00:00 2023-06-23T01:02:21.625 INFO:teuthology.orchestra.run.smithi152.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 28 MB/s | 605 kB 00:00 2023-06-23T01:02:21.639 INFO:teuthology.orchestra.run.smithi152.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 6.0 MB/s | 83 kB 00:00 2023-06-23T01:02:21.653 INFO:teuthology.orchestra.run.smithi152.stdout:(81/120): elfutils-libelf-devel-0.182-3.el8.x86 4.1 MB/s | 59 kB 00:00 2023-06-23T01:02:21.667 INFO:teuthology.orchestra.run.smithi152.stdout:(82/120): libquadmath-8.4.1-1.el8.x86_64.rpm 12 MB/s | 169 kB 00:00 2023-06-23T01:02:21.680 INFO:teuthology.orchestra.run.smithi152.stdout:(83/120): libcom_err-devel-1.45.6-1.el8.x86_64. 3.0 MB/s | 38 kB 00:00 2023-06-23T01:02:21.693 INFO:teuthology.orchestra.run.smithi152.stdout:(84/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.6 MB/s | 86 kB 00:00 2023-06-23T01:02:21.707 INFO:teuthology.orchestra.run.smithi152.stdout:(85/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.0 MB/s | 58 kB 00:00 2023-06-23T01:02:21.720 INFO:teuthology.orchestra.run.smithi152.stdout:(86/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.7 MB/s | 47 kB 00:00 2023-06-23T01:02:21.742 INFO:teuthology.orchestra.run.smithi152.stdout:(87/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 25 MB/s | 559 kB 00:00 2023-06-23T01:02:21.763 INFO:teuthology.orchestra.run.smithi152.stdout:(88/120): libgfortran-8.4.1-1.el8.x86_64.rpm 30 MB/s | 639 kB 00:00 2023-06-23T01:02:21.781 INFO:teuthology.orchestra.run.smithi152.stdout:(89/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 3.8 MB/s | 70 kB 00:00 2023-06-23T01:02:21.801 INFO:teuthology.orchestra.run.smithi152.stdout:(90/120): libselinux-devel-2.9-5.el8.x86_64.rpm 10 MB/s | 200 kB 00:00 2023-06-23T01:02:21.816 INFO:teuthology.orchestra.run.smithi152.stdout:(91/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.1 MB/s | 97 kB 00:00 2023-06-23T01:02:22.024 INFO:teuthology.orchestra.run.smithi152.stdout:(92/120): python3-scipy-1.0.0-20.module+el8.1.0 70 MB/s | 15 MB 00:00 2023-06-23T01:02:22.038 INFO:teuthology.orchestra.run.smithi152.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 3.3 MB/s | 44 kB 00:00 2023-06-23T01:02:22.055 INFO:teuthology.orchestra.run.smithi152.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.5 MB/s | 27 kB 00:00 2023-06-23T01:02:22.074 INFO:teuthology.orchestra.run.smithi152.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 8.5 MB/s | 161 kB 00:00 2023-06-23T01:02:22.108 INFO:teuthology.orchestra.run.smithi152.stdout:(96/120): ceph-mgr-diskprediction-local-17.2.6- 3.1 MB/s | 7.4 MB 00:02 2023-06-23T01:02:22.113 INFO:teuthology.orchestra.run.smithi152.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 8.0 MB/s | 320 kB 00:00 2023-06-23T01:02:22.151 INFO:teuthology.orchestra.run.smithi152.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-06-23T01:02:22.168 INFO:teuthology.orchestra.run.smithi152.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.2 MB/s | 39 kB 00:00 2023-06-23T01:02:22.191 INFO:teuthology.orchestra.run.smithi152.stdout:(100/120): python3-babel-2.5.1-5.el8.noarch.rpm 62 MB/s | 4.8 MB 00:00 2023-06-23T01:02:22.199 INFO:teuthology.orchestra.run.smithi152.stdout:(101/120): bison-3.0.4-10.el8.x86_64.rpm 22 MB/s | 688 kB 00:00 2023-06-23T01:02:22.216 INFO:teuthology.orchestra.run.smithi152.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 11 MB/s | 259 kB 00:00 2023-06-23T01:02:22.228 INFO:teuthology.orchestra.run.smithi152.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 7.6 MB/s | 225 kB 00:00 2023-06-23T01:02:22.236 INFO:teuthology.orchestra.run.smithi152.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 5.7 MB/s | 126 kB 00:00 2023-06-23T01:02:22.253 INFO:teuthology.orchestra.run.smithi152.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 10 MB/s | 187 kB 00:00 2023-06-23T01:02:22.278 INFO:teuthology.orchestra.run.smithi152.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 21 MB/s | 538 kB 00:00 2023-06-23T01:02:22.293 INFO:teuthology.orchestra.run.smithi152.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 58 MB/s | 3.7 MB 00:00 2023-06-23T01:02:22.299 INFO:teuthology.orchestra.run.smithi152.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 15 MB/s | 302 kB 00:00 2023-06-23T01:02:22.313 INFO:teuthology.orchestra.run.smithi152.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 8.0 MB/s | 157 kB 00:00 2023-06-23T01:02:22.316 INFO:teuthology.orchestra.run.smithi152.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 6.4 MB/s | 110 kB 00:00 2023-06-23T01:02:22.332 INFO:teuthology.orchestra.run.smithi152.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 18 MB/s | 353 kB 00:00 2023-06-23T01:02:22.333 INFO:teuthology.orchestra.run.smithi152.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.2 MB/s | 20 kB 00:00 2023-06-23T01:02:22.350 INFO:teuthology.orchestra.run.smithi152.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.1 MB/s | 54 kB 00:00 2023-06-23T01:02:22.357 INFO:teuthology.orchestra.run.smithi152.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 28 MB/s | 709 kB 00:00 2023-06-23T01:02:22.428 INFO:teuthology.orchestra.run.smithi152.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 61 MB/s | 4.7 MB 00:00 2023-06-23T01:02:22.446 INFO:teuthology.orchestra.run.smithi152.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 52 MB/s | 4.6 MB 00:00 2023-06-23T01:02:22.454 INFO:teuthology.orchestra.run.smithi152.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 26 MB/s | 713 kB 00:00 2023-06-23T01:02:23.143 INFO:teuthology.orchestra.run.smithi152.stdout:(118/120): librbd1-17.2.6-719.g42ddc46e.el8.x86 5.2 MB/s | 3.6 MB 00:00 2023-06-23T01:02:23.598 INFO:teuthology.orchestra.run.smithi152.stdout:(119/120): ceph-test-17.2.6-719.g42ddc46e.el8.x 3.7 MB/s | 52 MB 00:13 2023-06-23T01:02:23.770 INFO:teuthology.orchestra.run.smithi152.stdout:(120/120): librados2-17.2.6-719.g42ddc46e.el8.x 2.6 MB/s | 3.5 MB 00:01 2023-06-23T01:02:23.776 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-06-23T01:02:23.776 INFO:teuthology.orchestra.run.smithi152.stdout:Total 9.2 MB/s | 201 MB 00:21 2023-06-23T01:02:23.829 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-06-23T01:02:23.945 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-06-23T01:02:23.946 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-06-23T01:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2023-06-23T01:02:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-23T01:02:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-06-23T01:02:25.727 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-06-23T01:02:25.731 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-06-23T01:02:25.829 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-06-23T01:02:25.972 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-06-23T01:02:26.220 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-06-23T01:02:26.631 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-06-23T01:02:26.853 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:02:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:02:27.561 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:02:27.729 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:02:27.912 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:02:28.040 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:02:28.385 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T01:02:29.120 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-23T01:02:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:02:29.437 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:02:29.674 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-23T01:02:29.877 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-23T01:02:30.070 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:02:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:02:30.336 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-06-23T01:02:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:02:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:02:30.572 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-06-23T01:02:30.586 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-06-23T01:02:30.748 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T01:02:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-06-23T01:02:30.893 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-06-23T01:02:30.896 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-06-23T01:02:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-06-23T01:02:31.123 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T01:02:31.509 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-06-23T01:02:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-06-23T01:02:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:02:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-06-23T01:02:32.143 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:02:32.356 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:02:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-06-23T01:02:32.479 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 5/122 2023-06-23T01:02:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-06-23T01:02:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:02:32.807 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:02:32.994 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:02:33.286 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-06-23T01:02:33.379 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-06-23T01:02:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-06-23T01:02:33.894 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-06-23T01:02:33.956 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-06-23T01:02:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-06-23T01:02:34.114 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:02:34.223 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:02:34.393 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-06-23T01:02:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-06-23T01:02:34.672 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-06-23T01:02:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-06-23T01:02:34.889 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:02:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-23T01:02:34.981 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:02:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:02:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-23T01:02:35.184 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 14/122 2023-06-23T01:02:35.281 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-23T01:02:35.385 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-06-23T01:02:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:02:35.636 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-06-23T01:02:35.896 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-06-23T01:02:36.050 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-06-23T01:02:36.180 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T01:02:36.332 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-06-23T01:02:36.505 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T01:02:36.607 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-06-23T01:02:36.980 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-06-23T01:02:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-06-23T01:02:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T01:02:37.273 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-06-23T01:02:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-06-23T01:02:37.761 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-06-23T01:02:37.947 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T01:02:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-06-23T01:02:38.326 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-06-23T01:02:38.695 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-06-23T01:02:38.958 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-06-23T01:02:39.231 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-06-23T01:02:39.484 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-06-23T01:02:39.670 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 26/122 2023-06-23T01:02:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 27/122 2023-06-23T01:02:40.022 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:02:40.253 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-06-23T01:02:40.496 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-06-23T01:02:40.500 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-06-23T01:02:40.679 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T01:02:40.916 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-06-23T01:02:41.169 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-23T01:02:41.388 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-23T01:02:41.580 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-23T01:02:41.782 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-23T01:02:41.970 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-23T01:02:42.168 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-06-23T01:02:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-06-23T01:02:42.528 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-06-23T01:02:42.704 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-06-23T01:02:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-06-23T01:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-06-23T01:02:43.045 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-06-23T01:02:43.120 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-06-23T01:02:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-06-23T01:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-06-23T01:02:43.874 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-06-23T01:02:44.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-06-23T01:02:44.134 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-06-23T01:02:44.450 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-06-23T01:02:44.457 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-06-23T01:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-06-23T01:02:44.699 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-06-23T01:02:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-06-23T01:02:44.943 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/122 2023-06-23T01:02:45.042 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-06-23T01:02:45.140 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-06-23T01:02:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-06-23T01:02:45.339 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 38/122 2023-06-23T01:02:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-06-23T01:02:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 39/122 2023-06-23T01:02:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-06-23T01:02:45.760 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-06-23T01:02:45.772 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 40/122 2023-06-23T01:02:45.937 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-06-23T01:02:45.961 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-06-23T01:02:46.128 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-23T01:02:46.144 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-06-23T01:02:46.318 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-23T01:02:46.325 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-06-23T01:02:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-06-23T01:02:46.586 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-23T01:02:46.687 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T01:02:46.763 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-06-23T01:02:46.794 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-23T01:02:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-06-23T01:02:46.993 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-23T01:02:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-06-23T01:02:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-23T01:02:47.277 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-06-23T01:02:47.365 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-23T01:02:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-06-23T01:02:47.568 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-23T01:02:47.605 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-06-23T01:02:47.789 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-06-23T01:02:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-23T01:02:47.923 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-06-23T01:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-23T01:02:48.084 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-06-23T01:02:48.239 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-06-23T01:02:48.263 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-23T01:02:48.452 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-23T01:02:48.534 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-06-23T01:02:48.725 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-23T01:02:48.776 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-06-23T01:02:48.973 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-23T01:02:49.052 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-06-23T01:02:49.198 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-23T01:02:49.366 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-06-23T01:02:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-23T01:02:49.511 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-06-23T01:02:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-06-23T01:02:49.700 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-23T01:02:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-23T01:02:49.924 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-06-23T01:02:50.128 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-06-23T01:02:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-23T01:02:50.318 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-06-23T01:02:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-06-23T01:02:50.664 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-06-23T01:02:50.844 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-06-23T01:02:51.098 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-06-23T01:02:51.313 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-06-23T01:02:51.418 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-23T01:02:51.522 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-06-23T01:02:51.725 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-06-23T01:02:51.750 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-23T01:02:51.962 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-06-23T01:02:52.012 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-23T01:02:52.207 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-06-23T01:02:52.352 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-23T01:02:52.408 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-06-23T01:02:52.625 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-23T01:02:52.692 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-06-23T01:02:52.940 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-23T01:02:52.951 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-06-23T01:02:53.131 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-23T01:02:53.134 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-06-23T01:02:53.361 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-23T01:02:53.383 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-06-23T01:02:53.583 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-23T01:02:54.516 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-06-23T01:02:54.808 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-06-23T01:02:55.055 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-06-23T01:02:55.413 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-06-23T01:02:55.706 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-06-23T01:02:56.035 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-06-23T01:02:56.704 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-06-23T01:02:56.894 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-06-23T01:02:57.086 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-06-23T01:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:02:59.499 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:03:00.133 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:03:00.405 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:03:00.484 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:03:00.624 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:03:02.091 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:03:02.318 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:03:02.922 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 87/122 2023-06-23T01:03:03.219 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:03:03.309 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 88/122 2023-06-23T01:03:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:03:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:03:19.019 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2023-06-23T01:03:19.019 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2023-06-23T01:03:19.019 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /dev 2023-06-23T01:03:19.019 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /run 2023-06-23T01:03:19.019 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2023-06-23T01:03:19.019 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2023-06-23T01:03:19.020 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2023-06-23T01:03:19.020 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2023-06-23T01:03:19.020 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /dev 2023-06-23T01:03:19.020 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:19.437 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:03:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:03:21.210 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:03:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:03:21.738 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 89/122 2023-06-23T01:03:21.738 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2023-06-23T01:03:21.739 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2023-06-23T01:03:21.739 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /dev 2023-06-23T01:03:21.739 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /run 2023-06-23T01:03:21.739 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2023-06-23T01:03:21.740 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2023-06-23T01:03:21.740 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2023-06-23T01:03:21.740 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2023-06-23T01:03:21.741 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /dev 2023-06-23T01:03:21.741 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:22.017 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:03:22.098 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:03:22.204 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:03:22.224 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:03:23.586 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 90/122 2023-06-23T01:03:23.749 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:03:23.971 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 91/122 2023-06-23T01:03:24.116 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:03:24.505 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 92/122 2023-06-23T01:03:24.656 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:03:24.673 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 93/122 2023-06-23T01:03:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:03:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:03:25.510 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-23T01:03:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-23T01:03:25.903 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:03:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:03:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:03:27.487 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:03:27.536 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:03:27.641 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:03:27.649 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 94/122 2023-06-23T01:03:27.820 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 95/122 2023-06-23T01:03:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 96/122 2023-06-23T01:03:28.040 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-23T01:03:28.197 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:03:28.218 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:03:28.256 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:03:28.412 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 97/122 2023-06-23T01:03:28.428 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:03:28.439 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:03:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:03:29.605 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 98/122 2023-06-23T01:03:29.797 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:03:29.896 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 99/122 2023-06-23T01:03:30.107 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-23T01:03:30.304 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 100/122 2023-06-23T01:03:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:03:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:03:30.473 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:03:30.510 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 101/122 2023-06-23T01:03:30.628 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:03:30.672 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:03:30.688 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:03:30.699 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 102/122 2023-06-23T01:03:30.852 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:03:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:03:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:03:32.285 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 103/122 2023-06-23T01:03:32.523 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:03:32.617 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 104/122 2023-06-23T01:03:32.780 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:03:32.823 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 105/122 2023-06-23T01:03:32.963 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:03:33.312 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 106/122 2023-06-23T01:03:33.479 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:03:36.644 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:03:36.989 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-23T01:03:37.126 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-06-23T01:03:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-06-23T01:03:38.266 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-06-23T01:03:38.415 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 107/122 2023-06-23T01:03:38.460 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 111/122 2023-06-23T01:03:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-06-23T01:03:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-23T01:03:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-23T01:03:39.111 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-23T01:03:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-23T01:03:39.356 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 108/122 2023-06-23T01:03:39.511 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-23T01:03:39.739 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-06-23T01:03:39.786 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-06-23T01:03:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-06-23T01:03:39.974 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 110/122 2023-06-23T01:03:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-06-23T01:03:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-06-23T01:03:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-06-23T01:03:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:03:40.189 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-06-23T01:03:40.361 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T01:03:40.373 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-06-23T01:03:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:03:40.404 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:03:40.549 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 113/122 2023-06-23T01:03:40.725 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-06-23T01:03:40.895 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 115/122 2023-06-23T01:03:40.964 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:03:40.965 INFO:teuthology.orchestra.run.smithi005.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T01:03:40.965 INFO:teuthology.orchestra.run.smithi005.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T01:03:40.966 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:41.065 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 116/122 2023-06-23T01:03:41.305 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-23T01:03:41.305 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-23T01:03:41.305 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-23T01:03:41.306 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:03:41.306 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-23T01:03:41.306 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:03:41.306 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:03:41.306 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T01:03:41.307 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-23T01:03:41.307 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:03:41.307 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-23T01:03:41.307 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-23T01:03:41.307 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:03:41.307 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-23T01:03:41.307 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-23T01:03:41.307 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-23T01:03:41.308 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-23T01:03:41.308 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-23T01:03:41.308 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-23T01:03:41.308 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-23T01:03:41.308 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-23T01:03:41.308 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-23T01:03:41.309 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-23T01:03:41.309 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-23T01:03:41.309 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-23T01:03:41.316 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-23T01:03:41.316 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-23T01:03:41.316 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:03:41.317 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-23T01:03:41.317 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-23T01:03:41.317 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-23T01:03:41.317 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-23T01:03:41.317 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-23T01:03:41.317 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-23T01:03:41.317 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-23T01:03:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-23T01:03:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-23T01:03:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-23T01:03:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-23T01:03:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-23T01:03:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-23T01:03:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-23T01:03:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-23T01:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-23T01:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-23T01:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-23T01:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-23T01:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-23T01:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-23T01:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-23T01:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-23T01:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-23T01:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-23T01:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-23T01:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-23T01:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-23T01:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-23T01:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-23T01:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-23T01:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-23T01:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-23T01:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-23T01:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-23T01:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-23T01:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-23T01:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-23T01:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-23T01:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-23T01:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-23T01:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-23T01:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-23T01:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-06-23T01:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-06-23T01:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-06-23T01:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-06-23T01:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-06-23T01:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-06-23T01:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-06-23T01:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-06-23T01:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-06-23T01:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-06-23T01:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-06-23T01:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-06-23T01:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-06-23T01:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-06-23T01:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-06-23T01:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-06-23T01:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-06-23T01:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-06-23T01:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-06-23T01:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-06-23T01:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-06-23T01:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-06-23T01:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-06-23T01:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-06-23T01:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-06-23T01:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-06-23T01:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-06-23T01:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-06-23T01:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-06-23T01:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-06-23T01:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-06-23T01:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-06-23T01:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-06-23T01:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-06-23T01:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-06-23T01:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-06-23T01:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-06-23T01:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-06-23T01:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-06-23T01:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-06-23T01:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-06-23T01:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-06-23T01:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-06-23T01:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-06-23T01:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-06-23T01:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-06-23T01:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-06-23T01:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-23T01:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-23T01:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-23T01:03:41.330 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 117/122 2023-06-23T01:03:41.503 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T01:03:41.565 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-06-23T01:03:41.737 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T01:03:41.745 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-06-23T01:03:41.905 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-06-23T01:03:41.906 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:03:42.131 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-06-23T01:03:42.131 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:03:42.682 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:03:42.683 INFO:teuthology.orchestra.run.smithi152.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T01:03:42.683 INFO:teuthology.orchestra.run.smithi152.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T01:03:42.683 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:43.007 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 1/122 2023-06-23T01:03:43.008 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2/122 2023-06-23T01:03:43.008 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 3/122 2023-06-23T01:03:43.008 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 4/122 2023-06-23T01:03:43.008 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-719.g42ddc4 5/122 2023-06-23T01:03:43.008 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 6/122 2023-06-23T01:03:43.009 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 7/122 2023-06-23T01:03:43.009 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 8/122 2023-06-23T01:03:43.009 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 9/122 2023-06-23T01:03:43.009 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 10/122 2023-06-23T01:03:43.009 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 11/122 2023-06-23T01:03:43.009 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 12/122 2023-06-23T01:03:43.009 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 13/122 2023-06-23T01:03:43.010 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_6 14/122 2023-06-23T01:03:43.010 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 15/122 2023-06-23T01:03:43.010 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 16/122 2023-06-23T01:03:43.010 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 17/122 2023-06-23T01:03:43.010 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_ 18/122 2023-06-23T01:03:43.010 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 19/122 2023-06-23T01:03:43.010 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8 20/122 2023-06-23T01:03:43.011 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x 21/122 2023-06-23T01:03:43.011 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 22/122 2023-06-23T01:03:43.011 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 23/122 2023-06-23T01:03:43.011 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 24/122 2023-06-23T01:03:43.011 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 25/122 2023-06-23T01:03:43.011 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 26/122 2023-06-23T01:03:43.011 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 27/122 2023-06-23T01:03:43.012 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 28/122 2023-06-23T01:03:43.012 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.e 29/122 2023-06-23T01:03:43.012 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noar 30/122 2023-06-23T01:03:43.012 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.no 31/122 2023-06-23T01:03:43.012 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-719.g42dd 32/122 2023-06-23T01:03:43.012 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8 33/122 2023-06-23T01:03:43.012 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 34/122 2023-06-23T01:03:43.013 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el 35/122 2023-06-23T01:03:43.013 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 36/122 2023-06-23T01:03:43.013 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 37/122 2023-06-23T01:03:43.013 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 38/122 2023-06-23T01:03:43.013 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-06-23T01:03:43.013 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-06-23T01:03:43.013 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-06-23T01:03:43.014 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-06-23T01:03:43.014 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-06-23T01:03:43.014 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-06-23T01:03:43.014 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-06-23T01:03:43.014 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-06-23T01:03:43.014 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-06-23T01:03:43.014 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-06-23T01:03:43.015 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-06-23T01:03:43.015 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-06-23T01:03:43.015 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-06-23T01:03:43.015 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-06-23T01:03:43.016 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-06-23T01:03:43.016 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-06-23T01:03:43.016 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-06-23T01:03:43.016 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-06-23T01:03:43.017 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-06-23T01:03:43.017 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-06-23T01:03:43.017 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-06-23T01:03:43.017 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/122 2023-06-23T01:03:43.018 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-06-23T01:03:43.018 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-06-23T01:03:43.018 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-06-23T01:03:43.019 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-06-23T01:03:43.019 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-06-23T01:03:43.019 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-06-23T01:03:43.020 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-06-23T01:03:43.020 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-06-23T01:03:43.020 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-06-23T01:03:43.021 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-06-23T01:03:43.021 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-06-23T01:03:43.021 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-06-23T01:03:43.021 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-06-23T01:03:43.022 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-06-23T01:03:43.022 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-06-23T01:03:43.022 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-06-23T01:03:43.023 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-06-23T01:03:43.023 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-06-23T01:03:43.023 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-06-23T01:03:43.024 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-06-23T01:03:43.024 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-06-23T01:03:43.024 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-06-23T01:03:43.024 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-06-23T01:03:43.025 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-06-23T01:03:43.025 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-06-23T01:03:43.025 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-06-23T01:03:43.025 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-06-23T01:03:43.026 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-06-23T01:03:43.026 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-06-23T01:03:43.026 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-06-23T01:03:43.027 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-06-23T01:03:43.027 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-06-23T01:03:43.027 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-06-23T01:03:43.027 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-06-23T01:03:43.027 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-06-23T01:03:43.028 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-06-23T01:03:43.028 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-06-23T01:03:43.028 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-06-23T01:03:43.028 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-06-23T01:03:43.028 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-06-23T01:03:43.028 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-06-23T01:03:43.029 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-06-23T01:03:43.029 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-06-23T01:03:43.029 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-06-23T01:03:43.029 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-06-23T01:03:43.029 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-06-23T01:03:43.029 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-06-23T01:03:43.030 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-06-23T01:03:43.030 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-06-23T01:03:43.030 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-06-23T01:03:43.030 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-06-23T01:03:43.030 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-06-23T01:03:43.030 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-06-23T01:03:43.030 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-06-23T01:03:43.031 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-06-23T01:03:43.031 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-06-23T01:03:43.031 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-06-23T01:03:43.031 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-06-23T01:03:43.031 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 119/122 2023-06-23T01:03:43.031 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-06-23T01:03:43.031 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 121/122 2023-06-23T01:03:43.598 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:03:43.598 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:53 ago on Fri 23 Jun 2023 01:01:50 AM UTC. 2023-06-23T01:03:45.165 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-06-23T01:03:45.166 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:52 ago on Fri 23 Jun 2023 01:01:53 AM UTC. 2023-06-23T01:03:45.787 INFO:teuthology.orchestra.run.smithi005.stdout:Installed products updated. 2023-06-23T01:03:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:45.945 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2023-06-23T01:03:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.945 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.946 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2023-06-23T01:03:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T01:03:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T01:03:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.946 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.948 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-23T01:03:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T01:03:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-23T01:03:45.949 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T01:03:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: jq-1.5-12.el8.x86_64 2023-06-23T01:03:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-23T01:03:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-23T01:03:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T01:03:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.950 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-23T01:03:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-23T01:03:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T01:03:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-23T01:03:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T01:03:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.951 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-23T01:03:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T01:03:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-23T01:03:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T01:03:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T01:03:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-23T01:03:45.952 INFO:teuthology.orchestra.run.smithi005.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T01:03:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T01:03:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T01:03:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-23T01:03:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-23T01:03:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-23T01:03:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T01:03:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T01:03:45.953 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T01:03:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T01:03:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-23T01:03:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T01:03:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T01:03:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T01:03:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.954 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T01:03:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T01:03:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T01:03:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T01:03:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T01:03:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T01:03:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-23T01:03:45.955 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T01:03:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T01:03:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T01:03:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T01:03:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T01:03:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T01:03:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T01:03:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T01:03:45.956 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T01:03:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-23T01:03:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-23T01:03:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T01:03:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T01:03:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T01:03:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T01:03:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T01:03:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T01:03:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T01:03:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T01:03:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T01:03:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.958 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T01:03:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T01:03:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-23T01:03:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T01:03:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T01:03:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T01:03:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T01:03:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T01:03:45.959 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T01:03:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T01:03:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T01:03:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T01:03:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T01:03:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:45.960 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-23T01:03:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T01:03:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-23T01:03:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T01:03:45.961 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-23T01:03:45.962 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:45.962 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-06-23T01:03:46.194 DEBUG:teuthology.parallel:result is None 2023-06-23T01:03:47.298 INFO:teuthology.orchestra.run.smithi152.stdout:Installed products updated. 2023-06-23T01:03:47.437 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:47.437 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2023-06-23T01:03:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.438 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-06-23T01:03:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T01:03:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T01:03:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.438 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.439 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.440 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:17.2.6-719.g42ddc46e.el8.noarch 2023-06-23T01:03:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-06-23T01:03:47.441 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T01:03:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.7-12.el8.x86_64 2023-06-23T01:03:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T01:03:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: jq-1.5-12.el8.x86_64 2023-06-23T01:03:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-06-23T01:03:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-06-23T01:03:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T01:03:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.442 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-06-23T01:03:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-06-23T01:03:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T01:03:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-06-23T01:03:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T01:03:47.443 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-06-23T01:03:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T01:03:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-06-23T01:03:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T01:03:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T01:03:47.444 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-06-23T01:03:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T01:03:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T01:03:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T01:03:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.12-1.el8.x86_64 2023-06-23T01:03:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-06-23T01:03:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-06-23T01:03:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T01:03:47.445 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T01:03:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T01:03:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T01:03:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.5.1-5.el8.noarch 2023-06-23T01:03:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T01:03:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T01:03:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T01:03:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.446 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T01:03:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T01:03:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T01:03:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T01:03:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T01:03:47.447 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T01:03:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-06-23T01:03:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T01:03:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T01:03:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T01:03:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T01:03:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T01:03:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T01:03:47.448 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T01:03:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T01:03:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T01:03:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-06-23T01:03:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-06-23T01:03:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T01:03:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T01:03:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T01:03:47.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T01:03:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T01:03:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T01:03:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T01:03:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T01:03:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T01:03:47.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T01:03:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T01:03:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-06-23T01:03:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T01:03:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T01:03:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T01:03:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T01:03:47.451 INFO:teuthology.orchestra.run.smithi152.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T01:03:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T01:03:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T01:03:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T01:03:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T01:03:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T01:03:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.452 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:17.2.6-719.g42ddc46e.el8.x86_64 2023-06-23T01:03:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.3.3-2.el8.x86_64 2023-06-23T01:03:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T01:03:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-06-23T01:03:47.453 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T01:03:47.454 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-06-23T01:03:47.454 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:47.454 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-06-23T01:03:47.708 DEBUG:teuthology.parallel:result is None 2023-06-23T01:03:47.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:03:47.846 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T01:03:47.871 INFO:teuthology.orchestra.run.smithi005.stdout:17.2.6-719.g42ddc46e.el8 2023-06-23T01:03:47.872 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-23T01:03:47.872 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-23T01:03:47.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=42ddc46e6e6582d8e29a5a982a81886fe028be2c 2023-06-23T01:03:48.090 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T01:03:48.113 INFO:teuthology.orchestra.run.smithi152.stdout:17.2.6-719.g42ddc46e.el8 2023-06-23T01:03:48.114 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-719.g42ddc46e.el8 2023-06-23T01:03:48.114 INFO:teuthology.task.install:The correct ceph version 17.2.6-719.g42ddc46e is installed. 2023-06-23T01:03:48.114 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-23T01:03:48.115 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:48.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T01:03:48.160 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:03:48.160 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T01:03:48.201 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-23T01:03:48.201 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:48.202 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T01:03:48.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T01:03:48.297 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:03:48.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T01:03:48.322 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T01:03:48.387 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-23T01:03:48.387 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:48.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T01:03:48.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T01:03:48.480 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:03:48.481 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T01:03:48.505 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T01:03:48.569 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-23T01:03:48.570 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:48.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T01:03:48.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T01:03:48.661 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:03:48.661 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T01:03:48.686 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T01:03:48.750 INFO:teuthology.run_tasks:Running task ceph... 2023-06-23T01:03:48.831 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-23T01:03:48.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /var/log/ceph 2023-06-23T01:03:48.834 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /var/log/ceph 2023-06-23T01:03:48.858 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-23T01:03:48.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-23T01:03:48.899 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-23T01:03:48.923 INFO:tasks.ceph:Creating extra log directories... 2023-06-23T01:03:48.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-23T01:03:48.968 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-23T01:03:48.999 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-23T01:03:48.999 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-23T01:03:48.999 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-23T01:03:49.000 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:49.000 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-23T01:03:49.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-23T01:03:49.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-23T01:03:49.182 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:03:49.184 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-23T01:03:49.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-23T01:03:49.313 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-23T01:03:49.343 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-23T01:03:49.343 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-23T01:03:49.343 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927', 'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7311927', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9037, 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'fs', 'suite_branch': 'wip-yuri4-testing-2023-06-21-0846-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c', 'targets': {'smithi005.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOk9iHkStIKdX0e2hhQMYKWsnjcyeUlJFQaprlN9vcrlgJf1zAQW5lMrINiyEFNp3vlF1/GrAcyz+sP9MvS+cw/8LR0T9x0QvVQcfLoXOkTDBQ0cSA0C8V0980X82UkdwUSLQ7Rc4sUFVAUIArN7JKZlBlMljAhMc5Apt1JwYYScC7akB13fw5PgmoG3MalaMfjasoUy2Cxmxew5tWs/8EHRqE1MDShXdgHOpanXR/hvx8JIRhVTOWkfb9X9aHNO1eLiedcd5nQ6Prp3CzU2O3ghJ/W4CGJ7fmB7YSW5b6b0nkgwuyjocXCCva5RIyYNlhk7FvKxMRe5oVZx7VtNG+SzAiSgqY9RjBgy3GK3rnkZXmi+UfAQuFGYXp9l5kzGe3L/NMkm7VC9QpV7ZNU5U/fAvEqVKw3fImJo9wzkoEp1s+PA6e8VVPB7nDRcwB8Dy5XsMImCa4Be17I67BoUCym7fZk7drk9vi0NDk4ELyGcPQmWGgTdGckES8KbgtRys=', 'smithi152.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVIx9CZ40xo/rF3tqoJjc/54NIwjw2o4uiIp8TEbajXxRciQF9qUi141lvr2Mf8sIm7p605GD9L8u7Mq/N2iSE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '42ddc46e6e6582d8e29a5a982a81886fe028be2c'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '076bbebc42a14f7d568aaa78eabb0038327bcb23', 'timestamp': '2023-06-22_15:25:25', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-23T01:03:49.344 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-23T01:03:49.346 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-23T01:03:49.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/run/ceph 2023-06-23T01:03:49.402 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/run/ceph 2023-06-23T01:03:49.456 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:49.456 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T01:03:49.481 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T01:03:49.481 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2023-06-23T01:03:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T01:03:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:03:49.536 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 210661 Links: 1 2023-06-23T01:03:49.536 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:03:49.536 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:03:49.537 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-06-23 01:03:40.876675487 +0000 2023-06-23T01:03:49.537 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-06-23 01:00:59.780710679 +0000 2023-06-23T01:03:49.537 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-06-23 01:00:59.780710679 +0000 2023-06-23T01:03:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2023-06-23T01:03:49.537 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T01:03:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-06-23T01:03:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-06-23T01:03:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000103501 s, 4.9 MB/s 2023-06-23T01:03:49.600 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T01:03:49.656 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2023-06-23T01:03:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T01:03:49.711 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:03:49.711 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 210697 Links: 1 2023-06-23T01:03:49.712 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:03:49.712 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:03:49.712 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-06-23 01:03:40.876675487 +0000 2023-06-23T01:03:49.712 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-06-23 01:01:00.322699511 +0000 2023-06-23T01:03:49.712 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-06-23 01:01:00.322699511 +0000 2023-06-23T01:03:49.712 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2023-06-23T01:03:49.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T01:03:49.774 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-06-23T01:03:49.774 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-06-23T01:03:49.775 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000111576 s, 4.6 MB/s 2023-06-23T01:03:49.775 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T01:03:49.831 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2023-06-23T01:03:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T01:03:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:03:49.887 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 211821 Links: 1 2023-06-23T01:03:49.887 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:03:49.887 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:03:49.887 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-06-23 01:03:40.876675487 +0000 2023-06-23T01:03:49.887 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-06-23 01:01:00.873688157 +0000 2023-06-23T01:03:49.888 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-06-23 01:01:00.873688157 +0000 2023-06-23T01:03:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2023-06-23T01:03:49.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T01:03:49.950 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-06-23T01:03:49.950 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-06-23T01:03:49.950 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 9.2398e-05 s, 5.5 MB/s 2023-06-23T01:03:49.951 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T01:03:50.008 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2023-06-23T01:03:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T01:03:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:03:50.064 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 210769 Links: 1 2023-06-23T01:03:50.065 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:03:50.065 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:03:50.065 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-06-23 01:03:40.876675487 +0000 2023-06-23T01:03:50.065 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-06-23 01:01:01.399677319 +0000 2023-06-23T01:03:50.066 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-06-23 01:01:01.399677319 +0000 2023-06-23T01:03:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2023-06-23T01:03:50.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T01:03:50.127 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-06-23T01:03:50.127 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-06-23T01:03:50.128 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000110129 s, 4.6 MB/s 2023-06-23T01:03:50.129 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T01:03:50.185 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T01:03:50.185 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:03:50.186 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T01:03:50.202 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T01:03:50.202 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2023-06-23T01:03:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T01:03:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:03:50.258 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 210306 Links: 1 2023-06-23T01:03:50.258 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:03:50.258 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:03:50.258 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-06-23 01:03:42.596343202 +0000 2023-06-23T01:03:50.259 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-06-23 01:00:35.530519070 +0000 2023-06-23T01:03:50.259 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-06-23 01:00:35.530519070 +0000 2023-06-23T01:03:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2023-06-23T01:03:50.259 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T01:03:50.322 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-06-23T01:03:50.322 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-06-23T01:03:50.323 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.00025531 s, 2.0 MB/s 2023-06-23T01:03:50.323 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T01:03:50.380 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2023-06-23T01:03:50.435 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T01:03:50.435 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:03:50.435 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 209780 Links: 1 2023-06-23T01:03:50.436 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:03:50.436 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:03:50.436 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-06-23 01:03:42.596343202 +0000 2023-06-23T01:03:50.436 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-06-23 01:00:36.038513033 +0000 2023-06-23T01:03:50.436 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-06-23 01:00:36.038513033 +0000 2023-06-23T01:03:50.436 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2023-06-23T01:03:50.437 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T01:03:50.498 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-06-23T01:03:50.499 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-06-23T01:03:50.499 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000239256 s, 2.1 MB/s 2023-06-23T01:03:50.499 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T01:03:50.556 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2023-06-23T01:03:50.610 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T01:03:50.611 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:03:50.611 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 209903 Links: 1 2023-06-23T01:03:50.611 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:03:50.611 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:03:50.611 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-06-23 01:03:42.596343202 +0000 2023-06-23T01:03:50.612 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-06-23 01:00:36.520507305 +0000 2023-06-23T01:03:50.612 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-06-23 01:00:36.520507305 +0000 2023-06-23T01:03:50.612 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2023-06-23T01:03:50.612 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T01:03:50.673 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-06-23T01:03:50.674 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-06-23T01:03:50.674 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000250421 s, 2.0 MB/s 2023-06-23T01:03:50.675 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T01:03:50.730 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2023-06-23T01:03:50.784 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T01:03:50.785 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T01:03:50.785 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6h/6d Inode: 205804 Links: 1 2023-06-23T01:03:50.785 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T01:03:50.785 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T01:03:50.785 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-06-23 01:03:42.597343190 +0000 2023-06-23T01:03:50.786 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-06-23 01:00:37.021501351 +0000 2023-06-23T01:03:50.786 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-06-23 01:00:37.021501351 +0000 2023-06-23T01:03:50.786 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: - 2023-06-23T01:03:50.786 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T01:03:50.848 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-06-23T01:03:50.848 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-06-23T01:03:50.849 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000208282 s, 2.5 MB/s 2023-06-23T01:03:50.849 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T01:03:50.905 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T01:03:50.905 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-23T01:03:50.906 INFO:tasks.ceph:Generating config... 2023-06-23T01:03:50.907 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-23T01:03:50.907 INFO:tasks.ceph:[client] debug client = 20 2023-06-23T01:03:50.907 INFO:tasks.ceph:[client] debug ms = 1 2023-06-23T01:03:50.908 INFO:tasks.ceph:[client] pid file = 2023-06-23T01:03:50.908 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-23T01:03:50.908 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-23T01:03:50.908 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-23T01:03:50.909 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-23T01:03:50.909 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-23T01:03:50.909 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-23T01:03:50.910 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-23T01:03:50.910 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-23T01:03:50.910 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-23T01:03:50.910 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-23T01:03:50.911 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-23T01:03:50.911 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-23T01:03:50.911 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-23T01:03:50.911 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-23T01:03:50.912 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-23T01:03:50.912 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-23T01:03:50.912 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-23T01:03:50.912 INFO:tasks.ceph:[osd] bdev async discard = True 2023-06-23T01:03:50.912 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-06-23T01:03:50.913 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-06-23T01:03:50.913 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-23T01:03:50.913 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-23T01:03:50.913 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-06-23T01:03:50.914 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-06-23T01:03:50.914 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-23T01:03:50.914 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-23T01:03:50.914 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-06-23T01:03:50.915 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-23T01:03:50.915 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-23T01:03:50.915 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-23T01:03:50.915 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-23T01:03:50.915 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-23T01:03:50.916 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-23T01:03:50.916 INFO:tasks.ceph:Setting up mon.a... 2023-06-23T01:03:50.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-23T01:03:50.967 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.keyring 2023-06-23T01:03:50.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-23T01:03:51.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T01:03:51.076 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.5'), ('mon.b', '172.21.15.152'), ('mon.c', '[v2:172.21.15.152:3301,v1:172.21.15.152:6790]')] 2023-06-23T01:03:51.077 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.5,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-23T01:03:51.078 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:51.078 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-23T01:03:51.133 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.5 --add b 172.21.15.152 --addv c '[v2:172.21.15.152:3301,v1:172.21.15.152:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:03:51.216 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:03:51.216 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: generated fsid 002f2efe-a784-4dcf-91c2-b9f520cd2c9c 2023-06-23T01:03:51.217 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = octopus 2023-06-23T01:03:51.217 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 0 2023-06-23T01:03:51.217 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 002f2efe-a784-4dcf-91c2-b9f520cd2c9c 2023-06-23T01:03:51.217 INFO:teuthology.orchestra.run.smithi005.stdout:last_changed 2023-06-23T01:03:51.214066+0000 2023-06-23T01:03:51.217 INFO:teuthology.orchestra.run.smithi005.stdout:created 2023-06-23T01:03:51.214066+0000 2023-06-23T01:03:51.218 INFO:teuthology.orchestra.run.smithi005.stdout:min_mon_release 15 (octopus) 2023-06-23T01:03:51.218 INFO:teuthology.orchestra.run.smithi005.stdout:election_strategy: 1 2023-06-23T01:03:51.218 INFO:teuthology.orchestra.run.smithi005.stdout:0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2023-06-23T01:03:51.218 INFO:teuthology.orchestra.run.smithi005.stdout:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2023-06-23T01:03:51.218 INFO:teuthology.orchestra.run.smithi005.stdout:2: [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] mon.c 2023-06-23T01:03:51.219 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-23T01:03:51.220 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-23T01:03:51.275 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 002f2efe-a784-4dcf-91c2-b9f520cd2c9c... 2023-06-23T01:03:51.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-23T01:03:51.317 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-23T01:03:51.360 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2023-06-23T01:03:51.360 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2023-06-23T01:03:51.360 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-23T01:03:51.360 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2023-06-23T01:03:51.360 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.361 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2023-06-23T01:03:51.361 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.361 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2023-06-23T01:03:51.361 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.361 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2023-06-23T01:03:51.361 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2023-06-23T01:03:51.361 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.361 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2023-06-23T01:03:51.362 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2023-06-23T01:03:51.362 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.362 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-23T01:03:51.362 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2023-06-23T01:03:51.362 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.362 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2023-06-23T01:03:51.362 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-23T01:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2023-06-23T01:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2023-06-23T01:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2023-06-23T01:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2023-06-23T01:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2023-06-23T01:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2023-06-23T01:03:51.364 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2023-06-23T01:03:51.364 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2023-06-23T01:03:51.364 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-23T01:03:51.364 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-23T01:03:51.364 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2023-06-23T01:03:51.364 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.364 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-23T01:03:51.365 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.365 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2023-06-23T01:03:51.365 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.365 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2023-06-23T01:03:51.365 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.365 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2023-06-23T01:03:51.365 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2023-06-23T01:03:51.365 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2023-06-23T01:03:51.366 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2023-06-23T01:03:51.366 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 002f2efe-a784-4dcf-91c2-b9f520cd2c9c 2023-06-23T01:03:51.366 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.366 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2023-06-23T01:03:51.366 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2023-06-23T01:03:51.366 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.366 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2023-06-23T01:03:51.367 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2023-06-23T01:03:51.367 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2023-06-23T01:03:51.367 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.367 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2023-06-23T01:03:51.368 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2023-06-23T01:03:51.368 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.368 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2023-06-23T01:03:51.368 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2023-06-23T01:03:51.368 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2023-06-23T01:03:51.369 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.369 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2023-06-23T01:03:51.369 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2023-06-23T01:03:51.369 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.369 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2023-06-23T01:03:51.369 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.370 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2023-06-23T01:03:51.370 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.370 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2023-06-23T01:03:51.370 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.370 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2023-06-23T01:03:51.370 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2023-06-23T01:03:51.370 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.370 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2023-06-23T01:03:51.371 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2023-06-23T01:03:51.371 INFO:teuthology.orchestra.run.smithi005.stdout: bdev async discard = True 2023-06-23T01:03:51.371 INFO:teuthology.orchestra.run.smithi005.stdout: bdev enable discard = True 2023-06-23T01:03:51.371 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore allocator = bitmap 2023-06-23T01:03:51.371 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2023-06-23T01:03:51.371 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2023-06-23T01:03:51.371 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 1/20 2023-06-23T01:03:51.371 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 1/20 2023-06-23T01:03:51.372 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-06-23T01:03:51.372 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2023-06-23T01:03:51.372 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 4/10 2023-06-23T01:03:51.372 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2023-06-23T01:03:51.372 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2023-06-23T01:03:51.372 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2023-06-23T01:03:51.372 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2023-06-23T01:03:51.373 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2023-06-23T01:03:51.373 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2023-06-23T01:03:51.373 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.373 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2023-06-23T01:03:51.373 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-06-23T01:03:51.373 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2023-06-23T01:03:51.373 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2023-06-23T01:03:51.373 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2023-06-23T01:03:51.374 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2023-06-23T01:03:51.374 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2023-06-23T01:03:51.374 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2023-06-23T01:03:51.374 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.374 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2023-06-23T01:03:51.374 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-06-23T01:03:51.374 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2023-06-23T01:03:51.374 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2023-06-23T01:03:51.375 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2023-06-23T01:03:51.375 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2023-06-23T01:03:51.375 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2023-06-23T01:03:51.375 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2023-06-23T01:03:51.375 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2023-06-23T01:03:51.375 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2023-06-23T01:03:51.375 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2023-06-23T01:03:51.375 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.376 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-23T01:03:51.376 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2023-06-23T01:03:51.376 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2023-06-23T01:03:51.376 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.376 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2023-06-23T01:03:51.376 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-23T01:03:51.376 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-23T01:03:51.377 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.377 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2023-06-23T01:03:51.377 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2023-06-23T01:03:51.377 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.377 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2023-06-23T01:03:51.377 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2023-06-23T01:03:51.378 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:03:51.378 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2023-06-23T01:03:51.378 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2023-06-23T01:03:51.378 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2023-06-23T01:03:51.378 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2023-06-23T01:03:51.379 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-23T01:03:51.379 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-23T01:03:51.379 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2023-06-23T01:03:51.379 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2023-06-23T01:03:51.379 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-06-23T01:03:51.380 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2023-06-23T01:03:51.380 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2023-06-23T01:03:51.380 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2023-06-23T01:03:51.380 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2023-06-23T01:03:51.380 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2023-06-23T01:03:51.380 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2023-06-23T01:03:51.381 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2023-06-23T01:03:51.381 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2023-06-23T01:03:51.381 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2023-06-23T01:03:51.381 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2023-06-23T01:03:51.381 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2023-06-23T01:03:51.381 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2023-06-23T01:03:51.381 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-06-23T01:03:51.381 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2023-06-23T01:03:51.382 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2023-06-23T01:03:51.382 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2023-06-23T01:03:51.382 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2023-06-23T01:03:51.382 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2023-06-23T01:03:51.382 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2023-06-23T01:03:51.382 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2023-06-23T01:03:51.383 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2023-06-23T01:03:51.384 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2023-06-23T01:03:51.384 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-23T01:03:51.384 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2023-06-23T01:03:51.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.384 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2023-06-23T01:03:51.384 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2023-06-23T01:03:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2023-06-23T01:03:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2023-06-23T01:03:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2023-06-23T01:03:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2023-06-23T01:03:51.385 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.386 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-23T01:03:51.386 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2023-06-23T01:03:51.386 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.386 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2023-06-23T01:03:51.386 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-23T01:03:51.386 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.386 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2023-06-23T01:03:51.387 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2023-06-23T01:03:51.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2023-06-23T01:03:51.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2023-06-23T01:03:51.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2023-06-23T01:03:51.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2023-06-23T01:03:51.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2023-06-23T01:03:51.387 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2023-06-23T01:03:51.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-23T01:03:51.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-23T01:03:51.388 INFO:teuthology.orchestra.run.smithi152.stdout: mon_allow_pool_size_one = true 2023-06-23T01:03:51.388 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.388 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-23T01:03:51.389 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.389 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2023-06-23T01:03:51.389 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.389 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2023-06-23T01:03:51.389 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.389 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2023-06-23T01:03:51.390 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2023-06-23T01:03:51.390 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2023-06-23T01:03:51.390 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.5,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2023-06-23T01:03:51.390 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = 002f2efe-a784-4dcf-91c2-b9f520cd2c9c 2023-06-23T01:03:51.390 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.390 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2023-06-23T01:03:51.390 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2023-06-23T01:03:51.391 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.391 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2023-06-23T01:03:51.391 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 600 2023-06-23T01:03:51.391 INFO:teuthology.orchestra.run.smithi152.stdout: osd mclock profile = high_recovery_ops 2023-06-23T01:03:51.391 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.391 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2023-06-23T01:03:51.391 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2023-06-23T01:03:51.392 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.392 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2023-06-23T01:03:51.392 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2023-06-23T01:03:51.392 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2023-06-23T01:03:51.392 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.392 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2023-06-23T01:03:51.392 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2023-06-23T01:03:51.393 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.393 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2023-06-23T01:03:51.393 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.393 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2023-06-23T01:03:51.393 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.393 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2023-06-23T01:03:51.393 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.394 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2023-06-23T01:03:51.394 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2023-06-23T01:03:51.394 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.394 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2023-06-23T01:03:51.394 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2023-06-23T01:03:51.394 INFO:teuthology.orchestra.run.smithi152.stdout: bdev async discard = True 2023-06-23T01:03:51.394 INFO:teuthology.orchestra.run.smithi152.stdout: bdev enable discard = True 2023-06-23T01:03:51.394 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore allocator = bitmap 2023-06-23T01:03:51.395 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore block size = 96636764160 2023-06-23T01:03:51.395 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore fsck on mount = True 2023-06-23T01:03:51.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluefs = 1/20 2023-06-23T01:03:51.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluestore = 1/20 2023-06-23T01:03:51.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2023-06-23T01:03:51.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2023-06-23T01:03:51.395 INFO:teuthology.orchestra.run.smithi152.stdout: debug rocksdb = 4/10 2023-06-23T01:03:51.395 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd backfillfull_ratio = 0.85 2023-06-23T01:03:51.396 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd full ratio = 0.9 2023-06-23T01:03:51.396 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd nearfull ratio = 0.8 2023-06-23T01:03:51.396 INFO:teuthology.orchestra.run.smithi152.stdout: osd failsafe full ratio = 0.95 2023-06-23T01:03:51.396 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = bluestore 2023-06-23T01:03:51.396 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2023-06-23T01:03:51.396 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.396 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2023-06-23T01:03:51.397 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2023-06-23T01:03:51.397 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2023-06-23T01:03:51.397 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2023-06-23T01:03:51.397 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2023-06-23T01:03:51.397 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2023-06-23T01:03:51.397 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2023-06-23T01:03:51.397 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2023-06-23T01:03:51.397 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.398 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2023-06-23T01:03:51.398 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2023-06-23T01:03:51.398 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2023-06-23T01:03:51.398 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2023-06-23T01:03:51.398 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2023-06-23T01:03:51.398 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2023-06-23T01:03:51.399 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2023-06-23T01:03:51.399 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2023-06-23T01:03:51.399 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2023-06-23T01:03:51.399 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2023-06-23T01:03:51.399 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2023-06-23T01:03:51.399 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.399 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-23T01:03:51.400 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2023-06-23T01:03:51.400 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2023-06-23T01:03:51.400 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.400 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2023-06-23T01:03:51.400 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-23T01:03:51.400 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-23T01:03:51.401 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.401 INFO:teuthology.orchestra.run.smithi152.stdout: # 1m isn't quite enough 2023-06-23T01:03:51.401 INFO:teuthology.orchestra.run.smithi152.stdout: mon_down_mkfs_grace = 2m 2023-06-23T01:03:51.401 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.401 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_filestore_osds = false 2023-06-23T01:03:51.401 INFO:teuthology.orchestra.run.smithi152.stdout: mon op complaint time = 120 2023-06-23T01:03:51.401 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-06-23T01:03:51.402 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2023-06-23T01:03:51.402 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2023-06-23T01:03:51.402 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2023-06-23T01:03:51.402 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2023-06-23T01:03:51.402 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-23T01:03:51.402 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-23T01:03:51.402 INFO:teuthology.orchestra.run.smithi152.stdout: client mount timeout = 600 2023-06-23T01:03:51.402 INFO:teuthology.orchestra.run.smithi152.stdout: debug client = 20 2023-06-23T01:03:51.403 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2023-06-23T01:03:51.403 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = "" 2023-06-23T01:03:51.403 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 900 2023-06-23T01:03:51.403 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 900 2023-06-23T01:03:51.403 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2023-06-23T01:03:51.403 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2023-06-23T01:03:51.403 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2023-06-23T01:03:51.404 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2023-06-23T01:03:51.404 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.c] 2023-06-23T01:03:51.404 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2023-06-23T01:03:51.404 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.d] 2023-06-23T01:03:51.404 INFO:teuthology.orchestra.run.smithi152.stdout:[mds] 2023-06-23T01:03:51.404 INFO:teuthology.orchestra.run.smithi152.stdout: debug mds = 20 2023-06-23T01:03:51.404 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2023-06-23T01:03:51.404 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug frag = True 2023-06-23T01:03:51.405 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug scatterstat = True 2023-06-23T01:03:51.405 INFO:teuthology.orchestra.run.smithi152.stdout: mds op complaint time = 180 2023-06-23T01:03:51.405 INFO:teuthology.orchestra.run.smithi152.stdout: mds verify scatter = True 2023-06-23T01:03:51.405 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2023-06-23T01:03:51.405 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 900 2023-06-23T01:03:51.405 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 900 2023-06-23T01:03:51.406 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-23T01:03:51.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-23T01:03:51.476 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-23T01:03:51.476 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:51.476 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-23T01:03:51.491 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:51.492 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-23T01:03:51.545 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:03:51.545 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:51.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-23T01:03:51.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T01:03:51.620 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:03:51.620 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:03:51.675 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:03:51.676 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:03:51.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-23T01:03:51.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-23T01:03:51.711 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:03:51.712 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:03:51.766 INFO:tasks.ceph:Setting up mon nodes... 2023-06-23T01:03:51.766 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-23T01:03:51.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-23T01:03:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-23T01:03:51.818 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-23T01:03:51.876 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-23T01:03:51.878 INFO:tasks.ceph:Setting up mds nodes... 2023-06-23T01:03:51.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-23T01:03:51.928 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-23T01:03:51.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-23T01:03:51.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-23T01:03:52.044 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-23T01:03:52.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-23T01:03:52.072 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-23T01:03:52.121 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-23T01:03:52.124 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-23T01:03:52.148 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-23T01:03:52.237 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-23T01:03:52.239 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-23T01:03:52.264 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-23T01:03:52.265 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-23T01:03:52.304 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-23T01:03:52.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-06-23T01:03:52.397 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-06-23T01:03:52.410 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-23T01:03:52.448 INFO:teuthology.orchestra.run.smithi152.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-23T01:03:52.461 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-06-23T01:03:52.538 INFO:teuthology.orchestra.run.smithi152.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-06-23T01:03:52.551 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-23T01:03:52.551 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-23T01:03:52.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-23T01:03:52.576 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T01:03:52.577 INFO:tasks.ceph:role: osd.0 2023-06-23T01:03:52.577 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:03:52.577 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-23T01:03:52.640 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T01:03:52.641 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T01:03:52.641 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T01:03:52.641 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 2023-06-23T01:03:52.642 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T01:03:52.642 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2023-06-23T01:03:52.642 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T01:03:52.642 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T01:03:52.643 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T01:03:52.643 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T01:03:52.662 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2023-06-23T01:03:52.663 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2023-06-23T01:03:52.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-23T01:03:52.741 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-23T01:03:52.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-23T01:03:52.876 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T01:03:52.876 INFO:tasks.ceph:role: osd.1 2023-06-23T01:03:52.877 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:03:52.877 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-23T01:03:52.940 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T01:03:52.940 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T01:03:52.940 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T01:03:52.941 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 2023-06-23T01:03:52.941 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T01:03:52.941 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2023-06-23T01:03:52.941 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T01:03:52.941 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T01:03:52.941 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T01:03:52.941 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T01:03:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2023-06-23T01:03:52.964 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2023-06-23T01:03:52.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-23T01:03:53.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-23T01:03:53.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-23T01:03:53.175 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T01:03:53.176 INFO:tasks.ceph:role: osd.2 2023-06-23T01:03:53.176 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:03:53.176 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-23T01:03:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T01:03:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T01:03:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T01:03:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 2023-06-23T01:03:53.242 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T01:03:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2023-06-23T01:03:53.242 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T01:03:53.242 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T01:03:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T01:03:53.243 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T01:03:53.263 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2023-06-23T01:03:53.265 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2023-06-23T01:03:53.265 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-23T01:03:53.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-23T01:03:53.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-23T01:03:53.476 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-23T01:03:53.477 INFO:tasks.ceph:role: osd.3 2023-06-23T01:03:53.477 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:03:53.477 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-23T01:03:53.541 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T01:03:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T01:03:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T01:03:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 2023-06-23T01:03:53.542 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T01:03:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2023-06-23T01:03:53.542 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T01:03:53.542 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T01:03:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T01:03:53.543 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T01:03:53.564 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2023-06-23T01:03:53.565 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2023-06-23T01:03:53.565 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-23T01:03:53.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-23T01:03:53.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:03:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:53.961+0000 7fd7cf732540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-23T01:03:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:53.961+0000 7fd7cf732540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-23T01:03:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:53.961+0000 7fd7cf732540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T01:03:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:53.961+0000 7fd7cf732540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T01:03:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:53.961+0000 7fd7cf732540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-23T01:03:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:53.962+0000 7fd7cf732540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-23T01:03:54.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:54.484+0000 7fd7cf732540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:03:55.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:55.249+0000 7fd7cf732540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:03:56.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:56.010+0000 7fd7cf732540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:03:56.810 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-23T01:03:56.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:03:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:56.924+0000 7f5d3b2c8540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-23T01:03:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:56.925+0000 7f5d3b2c8540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-23T01:03:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:56.925+0000 7f5d3b2c8540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T01:03:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:56.925+0000 7f5d3b2c8540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T01:03:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:56.925+0000 7f5d3b2c8540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-23T01:03:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:56.925+0000 7f5d3b2c8540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-23T01:03:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:57.448+0000 7f5d3b2c8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:03:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:58.209+0000 7f5d3b2c8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:03:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:58.972+0000 7f5d3b2c8540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:03:59.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-23T01:03:59.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:03:59.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:59.883+0000 7ff9a25fc540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-23T01:03:59.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:59.883+0000 7ff9a25fc540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-23T01:03:59.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:59.884+0000 7ff9a25fc540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T01:03:59.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:59.884+0000 7ff9a25fc540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T01:03:59.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:59.884+0000 7ff9a25fc540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-23T01:03:59.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:03:59.884+0000 7ff9a25fc540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-23T01:04:00.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:00.408+0000 7ff9a25fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:01.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:01.172+0000 7ff9a25fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:01.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:01.935+0000 7ff9a25fc540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:02.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-23T01:04:02.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:04:02.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:02.852+0000 7f3602170540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-23T01:04:02.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:02.852+0000 7f3602170540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-23T01:04:02.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:02.852+0000 7f3602170540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T01:04:02.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:02.852+0000 7f3602170540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T01:04:02.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:02.852+0000 7f3602170540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-23T01:04:02.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:02.853+0000 7f3602170540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-23T01:04:03.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:03.375+0000 7f3602170540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:04.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:04.134+0000 7f3602170540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:04.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:04.897+0000 7f3602170540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:05.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-23T01:04:05.726 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-23T01:04:05.755 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T01:04:05.755 INFO:tasks.ceph:role: osd.4 2023-06-23T01:04:05.755 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:05.756 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-23T01:04:05.828 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T01:04:05.829 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T01:04:05.829 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T01:04:05.829 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 2023-06-23T01:04:05.829 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T01:04:05.829 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2023-06-23T01:04:05.829 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T01:04:05.830 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T01:04:05.830 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T01:04:05.830 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T01:04:06.269 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2023-06-23T01:04:06.270 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2023-06-23T01:04:06.271 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-23T01:04:06.444 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-23T01:04:06.478 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-23T01:04:06.544 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T01:04:06.544 INFO:tasks.ceph:role: osd.5 2023-06-23T01:04:06.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:06.545 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-23T01:04:06.616 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T01:04:06.616 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T01:04:06.617 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T01:04:06.617 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 2023-06-23T01:04:06.617 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T01:04:06.617 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2023-06-23T01:04:06.617 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T01:04:06.617 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T01:04:06.617 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T01:04:06.617 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T01:04:07.041 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2023-06-23T01:04:07.042 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2023-06-23T01:04:07.043 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-23T01:04:07.155 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-23T01:04:07.184 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-23T01:04:07.248 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T01:04:07.248 INFO:tasks.ceph:role: osd.6 2023-06-23T01:04:07.249 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:07.249 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-23T01:04:07.320 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T01:04:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T01:04:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T01:04:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 2023-06-23T01:04:07.321 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T01:04:07.321 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2023-06-23T01:04:07.322 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T01:04:07.322 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T01:04:07.322 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T01:04:07.322 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T01:04:07.762 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2023-06-23T01:04:07.763 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2023-06-23T01:04:07.763 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-23T01:04:07.910 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-23T01:04:07.938 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-23T01:04:08.003 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-23T01:04:08.003 INFO:tasks.ceph:role: osd.7 2023-06-23T01:04:08.003 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:08.004 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-23T01:04:08.074 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-23T01:04:08.075 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-23T01:04:08.075 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-23T01:04:08.075 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 2023-06-23T01:04:08.075 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-23T01:04:08.075 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2023-06-23T01:04:08.075 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-23T01:04:08.076 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-23T01:04:08.076 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-23T01:04:08.076 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-23T01:04:08.522 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2023-06-23T01:04:08.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2023-06-23T01:04:08.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-23T01:04:08.694 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-23T01:04:08.724 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:04:08.812 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:08.809+0000 7f317b872540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-23T01:04:08.812 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:08.809+0000 7f317b872540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-23T01:04:08.812 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:08.809+0000 7f317b872540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T01:04:08.812 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:08.809+0000 7f317b872540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T01:04:08.812 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:08.809+0000 7f317b872540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-23T01:04:08.813 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:08.811+0000 7f317b872540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-23T01:04:09.334 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:09.332+0000 7f317b872540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:10.096 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:10.094+0000 7f317b872540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:10.860 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:10.858+0000 7f317b872540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:11.662 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-23T01:04:11.691 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:04:11.777 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:11.773+0000 7f9006766540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-23T01:04:11.777 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:11.774+0000 7f9006766540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-23T01:04:11.777 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:11.774+0000 7f9006766540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T01:04:11.777 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:11.774+0000 7f9006766540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T01:04:11.778 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:11.774+0000 7f9006766540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-23T01:04:11.778 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:11.774+0000 7f9006766540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-23T01:04:12.304 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:12.301+0000 7f9006766540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:13.066 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:13.063+0000 7f9006766540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:13.829 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:13.826+0000 7f9006766540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:14.622 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-23T01:04:14.651 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:04:14.736 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:14.733+0000 7f8190697540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-23T01:04:14.737 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:14.733+0000 7f8190697540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-23T01:04:14.737 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:14.733+0000 7f8190697540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T01:04:14.737 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:14.733+0000 7f8190697540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T01:04:14.737 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:14.733+0000 7f8190697540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-23T01:04:14.737 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:14.734+0000 7f8190697540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-23T01:04:15.259 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:15.256+0000 7f8190697540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:16.019 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:16.016+0000 7f8190697540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:16.781 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:16.778+0000 7f8190697540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:17.578 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-23T01:04:17.605 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:04:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:17.683+0000 7f8440a12540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-23T01:04:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:17.684+0000 7f8440a12540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-23T01:04:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:17.684+0000 7f8440a12540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T01:04:17.687 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:17.684+0000 7f8440a12540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T01:04:17.688 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:17.684+0000 7f8440a12540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-23T01:04:17.688 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:17.684+0000 7f8440a12540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-23T01:04:18.208 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:18.205+0000 7f8440a12540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:18.974 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:18.971+0000 7f8440a12540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:19.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T01:04:19.349 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T01:04:19.736 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:19.733+0000 7f8440a12540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-06-23T01:04:20.525 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-23T01:04:20.551 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-23T01:04:20.551 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:20.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-23T01:04:20.580 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:20.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-23T01:04:20.649 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:20.650 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-23T01:04:20.716 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:20.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-23T01:04:20.782 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:20.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-23T01:04:20.848 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:20.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-23T01:04:20.915 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:20.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-23T01:04:20.980 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:20.981 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-23T01:04:21.005 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:21.006 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-23T01:04:21.069 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:21.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-23T01:04:21.133 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:21.133 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-23T01:04:21.196 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:21.197 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-23T01:04:21.260 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:21.261 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-23T01:04:21.324 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:21.325 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-23T01:04:21.388 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:21.388 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-23T01:04:21.404 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:21.405 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-06-23T01:04:21.459 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:21.460 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-23T01:04:21.474 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:21.474 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-06-23T01:04:21.528 INFO:tasks.ceph:Adding keys to all mons... 2023-06-23T01:04:21.528 DEBUG:teuthology.orchestra.run.smithi005:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-23T01:04:21.530 DEBUG:teuthology.orchestra.run.smithi152:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-23T01:04:21.570 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.y] 2023-06-23T01:04:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB375RkImuIMBAA308f9Mr3SYpbcOJAofSkwg== 2023-06-23T01:04:21.571 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2023-06-23T01:04:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB375Rku7k6NxAARtZTJIu9moLT/LoZyXB6Bw== 2023-06-23T01:04:21.571 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2023-06-23T01:04:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB475RkNKeFAhAAy6Hi4Lq+tru2NXO3cNniFQ== 2023-06-23T01:04:21.571 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.0] 2023-06-23T01:04:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB575RkF+hjORAA3Y/6rNYGjy1uPU3Wip53Dg== 2023-06-23T01:04:21.572 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.1] 2023-06-23T01:04:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB875Rk3hgzNxAAvc2KsfwSS5nYBq/htPlpNA== 2023-06-23T01:04:21.572 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.2] 2023-06-23T01:04:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB/75RkVo7ANBAASdausoI/rwcPc6ebDpi+ew== 2023-06-23T01:04:21.572 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.3] 2023-06-23T01:04:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCC75RkMHflMhAAW6XB3YjqxfscMYYAGrm7vw== 2023-06-23T01:04:21.572 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.x] 2023-06-23T01:04:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB375Rkht0hNBAAxlIS31Q0+SOetfcwZd8HEQ== 2023-06-23T01:04:21.573 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2023-06-23T01:04:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB475Rk+uklBxAAuJz6KnDrH0xkn/o6NQB/dQ== 2023-06-23T01:04:21.573 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2023-06-23T01:04:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB475Rk9y8PDhAAwc2vB5GdoQegLDGztcl4zQ== 2023-06-23T01:04:21.573 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.4] 2023-06-23T01:04:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCI75RkdSxFMBAA1armtWpTfF+pH2coTjUtXA== 2023-06-23T01:04:21.574 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.5] 2023-06-23T01:04:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCL75RkVVgwLhAALF3qw/6+onga+lZlO2NWFQ== 2023-06-23T01:04:21.574 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.6] 2023-06-23T01:04:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCO75Rki1TIKxAA55v2NVOhU6z77mdlwFqnZA== 2023-06-23T01:04:21.574 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.7] 2023-06-23T01:04:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCR75Rky2XVKBAATC226ouiwwjyWHMOlpUHJg== 2023-06-23T01:04:21.574 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2023-06-23T01:04:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB475RkkbAOEhAAYVdlNz5n3Q9/rYu6Zdhh8w== 2023-06-23T01:04:21.575 INFO:teuthology.orchestra.run.smithi005.stdout:[client.2] 2023-06-23T01:04:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB475Rk2yqPFxAAiIrw802Qow5kVzVFnXoMnQ== 2023-06-23T01:04:21.575 INFO:teuthology.orchestra.run.smithi005.stdout:[client.1] 2023-06-23T01:04:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB475RkTiSeGhAAFYZgeG+0EkJDku/D2g3djA== 2023-06-23T01:04:21.575 INFO:teuthology.orchestra.run.smithi005.stdout:[client.3] 2023-06-23T01:04:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQB475RkOoT8HxAA8OjDzsMtJbNl/Xm6yPi98Q== 2023-06-23T01:04:21.590 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.y] 2023-06-23T01:04:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB375RkImuIMBAA308f9Mr3SYpbcOJAofSkwg== 2023-06-23T01:04:21.591 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2023-06-23T01:04:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB375Rku7k6NxAARtZTJIu9moLT/LoZyXB6Bw== 2023-06-23T01:04:21.591 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.c] 2023-06-23T01:04:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB475RkNKeFAhAAy6Hi4Lq+tru2NXO3cNniFQ== 2023-06-23T01:04:21.591 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.0] 2023-06-23T01:04:21.591 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB575RkF+hjORAA3Y/6rNYGjy1uPU3Wip53Dg== 2023-06-23T01:04:21.591 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.1] 2023-06-23T01:04:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB875Rk3hgzNxAAvc2KsfwSS5nYBq/htPlpNA== 2023-06-23T01:04:21.592 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.2] 2023-06-23T01:04:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB/75RkVo7ANBAASdausoI/rwcPc6ebDpi+ew== 2023-06-23T01:04:21.592 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.3] 2023-06-23T01:04:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCC75RkMHflMhAAW6XB3YjqxfscMYYAGrm7vw== 2023-06-23T01:04:21.592 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.x] 2023-06-23T01:04:21.592 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB375Rkht0hNBAAxlIS31Q0+SOetfcwZd8HEQ== 2023-06-23T01:04:21.593 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2023-06-23T01:04:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB475Rk+uklBxAAuJz6KnDrH0xkn/o6NQB/dQ== 2023-06-23T01:04:21.593 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.d] 2023-06-23T01:04:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB475Rk9y8PDhAAwc2vB5GdoQegLDGztcl4zQ== 2023-06-23T01:04:21.593 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.4] 2023-06-23T01:04:21.593 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCI75RkdSxFMBAA1armtWpTfF+pH2coTjUtXA== 2023-06-23T01:04:21.593 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.5] 2023-06-23T01:04:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCL75RkVVgwLhAALF3qw/6+onga+lZlO2NWFQ== 2023-06-23T01:04:21.594 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.6] 2023-06-23T01:04:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCO75Rki1TIKxAA55v2NVOhU6z77mdlwFqnZA== 2023-06-23T01:04:21.594 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.7] 2023-06-23T01:04:21.594 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCR75Rky2XVKBAATC226ouiwwjyWHMOlpUHJg== 2023-06-23T01:04:21.594 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2023-06-23T01:04:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB475RkkbAOEhAAYVdlNz5n3Q9/rYu6Zdhh8w== 2023-06-23T01:04:21.595 INFO:teuthology.orchestra.run.smithi152.stdout:[client.2] 2023-06-23T01:04:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB475Rk2yqPFxAAiIrw802Qow5kVzVFnXoMnQ== 2023-06-23T01:04:21.595 INFO:teuthology.orchestra.run.smithi152.stdout:[client.1] 2023-06-23T01:04:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB475RkTiSeGhAAFYZgeG+0EkJDku/D2g3djA== 2023-06-23T01:04:21.595 INFO:teuthology.orchestra.run.smithi152.stdout:[client.3] 2023-06-23T01:04:21.595 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB475RkOoT8HxAA8OjDzsMtJbNl/Xm6yPi98Q== 2023-06-23T01:04:21.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T01:04:21.618 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T01:04:21.685 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T01:04:21.687 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T01:04:21.737 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T01:04:21.740 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T01:04:21.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:21.795 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:21.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:21.844 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:21.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:21.894 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:21.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:21.943 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:21.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T01:04:21.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-23T01:04:22.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T01:04:22.041 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T01:04:22.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T01:04:22.093 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-23T01:04:22.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:22.145 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:22.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:22.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:22.242 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:22.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:22.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:22.293 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-23T01:04:22.340 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T01:04:22.343 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T01:04:22.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T01:04:22.394 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T01:04:22.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T01:04:22.447 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T01:04:22.496 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T01:04:22.499 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-23T01:04:22.549 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-23T01:04:22.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-23T01:04:22.574 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T01:04:22.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-23T01:04:22.691 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-23T01:04:22.715 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T01:04:22.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-23T01:04:22.832 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-23T01:04:22.896 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-23T01:04:22.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-23T01:04:23.016 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:04:23.019 DEBUG:teuthology.orchestra.run.smithi152:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-23T01:04:23.070 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-23T01:04:23.070 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-23T01:04:23.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-23T01:04:23.074 INFO:tasks.ceph.mon.a:Started 2023-06-23T01:04:23.074 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-23T01:04:23.074 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-23T01:04:23.112 INFO:tasks.ceph.mon.b:Started 2023-06-23T01:04:23.112 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-23T01:04:23.113 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-23T01:04:23.114 INFO:tasks.ceph.mon.c:Started 2023-06-23T01:04:23.115 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-23T01:04:23.115 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-23T01:04:23.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-23T01:04:23.117 INFO:tasks.ceph.mgr.y:Started 2023-06-23T01:04:23.117 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-23T01:04:23.118 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-23T01:04:23.119 INFO:tasks.ceph.mgr.x:Started 2023-06-23T01:04:23.120 INFO:tasks.ceph:Setting crush tunables to default 2023-06-23T01:04:23.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd crush tunables default 2023-06-23T01:04:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.237+0000 7f151fcc3700 1 Processor -- start 2023-06-23T01:04:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.238+0000 7f151fcc3700 1 -- start start 2023-06-23T01:04:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.238+0000 7f151fcc3700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.238+0000 7f151fcc3700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151815d7a0 con 0x7f1518156ea0 2023-06-23T01:04:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.238+0000 7f151fcc3700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15180aa210 con 0x7f1518157270 2023-06-23T01:04:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.238+0000 7f151fcc3700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15181576e0 con 0x7f15180a9890 2023-06-23T01:04:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.239+0000 7f151da5f700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1518156ea0 0x7f151815dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:42748/0 (socket says 172.21.15.5:42748) 2023-06-23T01:04:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.239+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 learned_addr learned my addr 172.21.15.5:0/2017682628 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.239+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.239+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-23T01:04:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.239+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.239+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.239+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:04:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.440+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.440+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-23T01:04:23.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.440+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.440+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.440+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T01:04:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.841+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.841+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-23T01:04:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.841+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.841+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:23.841+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-23T01:04:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:24.642+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:24.642+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-23T01:04:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:24.642+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:24.642+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:24.642+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-23T01:04:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 msgr2=0x7f15180a9cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T01:04:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f150f7fe700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.5:3300/0 conn(0x7f15180a9890 0x7f15180a9cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 >> v1:172.21.15.152:6789/0 conn(0x7f1518157270 legacy=0x7f1518159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 >> v1:172.21.15.5:6789/0 conn(0x7f1518156ea0 legacy=0x7f151815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f150f7fe700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f150f7fe700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15181576e0 con 0x7f14fc006230 2023-06-23T01:04:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc008d50 con 0x7f14fc003780 2023-06-23T01:04:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-23T01:04:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-23T01:04:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:04:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:26.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.239+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T01:04:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-23T01:04:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T01:04:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-23T01:04:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.440+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T01:04:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-23T01:04:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:26.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-23T01:04:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-23T01:04:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:26.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:26.841+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-23T01:04:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-23T01:04:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151da5f700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-23T01:04:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-23T01:04:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-23T01:04:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-06-23T01:04:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:27.643+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-23T01:04:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T01:04:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f150f7fe700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 msgr2=0x7f14fc0086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T01:04:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f150f7fe700 1 --2- 172.21.15.5:0/2017682628 >> v2:172.21.15.152:3300/0 conn(0x7f14fc006230 0x7f14fc0086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f150f7fe700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 reap_dead start 2023-06-23T01:04:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151815a780 con 0x7f14fc006690 2023-06-23T01:04:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14fc00f010 con 0x7f1518157270 2023-06-23T01:04:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f150f7fe700 1 -- 172.21.15.5:0/2017682628 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15181576e0 con 0x7f14fc003780 2023-06-23T01:04:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121688177 0 0) 0x7f151815a780 con 0x7f14fc006690 2023-06-23T01:04:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.239+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1504003570 con 0x7f14fc006690 2023-06-23T01:04:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113210862 0 0) 0x7f14fc00f010 con 0x7f1518157270 2023-06-23T01:04:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f151815a780 con 0x7f1518157270 2023-06-23T01:04:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2846564085 0 0) 0x7f1504003570 con 0x7f14fc006690 2023-06-23T01:04:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f14fc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f14fc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 >> v1:172.21.15.152:6789/0 conn(0x7f1518157270 legacy=0x7f14fc00b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc010240 con 0x7f14fc006690 2023-06-23T01:04:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1394223029 0 0) 0x7f15100097a0 con 0x7f14fc006690 2023-06-23T01:04:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 >> v1:172.21.15.5:6789/0 conn(0x7f14fc006690 legacy=0x7f14fc00c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f1504003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1504004180 0x7f1504004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504004d50 0x7f15040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15181576e0 con 0x7f1504004180 2023-06-23T01:04:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150400bbf0 con 0x7f1504004d50 2023-06-23T01:04:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150400bd80 con 0x7f14fc003780 2023-06-23T01:04:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1504004180 0x7f1504004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f1504003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504004d50 0x7f15040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 msgr2=0x7f1504003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f1504003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504004d50 msgr2=0x7f15040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504004d50 0x7f15040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151d25e700 1 -- 172.21.15.5:0/2017682628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504003570 con 0x7f1504004180 2023-06-23T01:04:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151e260700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504004d50 0x7f15040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:04:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151da5f700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f14fc003780 0x7f1504003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:04:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.240+0000 7f151d25e700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1504004180 0x7f1504004810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1514003b20 tx=0x7f1514009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a946eb4c390133e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514006820 con 0x7f1504004180 2023-06-23T01:04:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514006980 con 0x7f1504004180 2023-06-23T01:04:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151ca5d700 1 -- 172.21.15.5:0/2017682628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514006b10 con 0x7f1504004180 2023-06-23T01:04:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151fcc3700 1 -- 172.21.15.5:0/2017682628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1504004180 msgr2=0x7f1504004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151fcc3700 1 --2- 172.21.15.5:0/2017682628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1504004180 0x7f1504004810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1514003b20 tx=0x7f1514009040 comp rx=0 tx=0).stop 2023-06-23T01:04:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151e260700 1 -- 172.21.15.5:0/2017682628 reap_dead start 2023-06-23T01:04:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151fcc3700 1 -- 172.21.15.5:0/2017682628 shutdown_connections 2023-06-23T01:04:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151fcc3700 1 -- 172.21.15.5:0/2017682628 >> 172.21.15.5:0/2017682628 conn(0x7f151806e830 msgr2=0x7f151806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151fcc3700 1 -- 172.21.15.5:0/2017682628 shutdown_connections 2023-06-23T01:04:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.241+0000 7f151fcc3700 1 -- 172.21.15.5:0/2017682628 wait complete. 2023-06-23T01:04:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151fcc3700 1 Processor -- start 2023-06-23T01:04:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151fcc3700 1 -- start start 2023-06-23T01:04:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151fcc3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1504004180 0x7f15181453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151fcc3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f151816e130 0x7f15180af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151fcc3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15180ca670 0x7f15180cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151fcc3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180cb050 con 0x7f151816e130 2023-06-23T01:04:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151fcc3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1518169f20 con 0x7f15180ca670 2023-06-23T01:04:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151fcc3700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151816adc0 con 0x7f1504004180 2023-06-23T01:04:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151d25e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f151816e130 0x7f15180af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151d25e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f151816e130 0x7f15180af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49544/0 (socket says 172.21.15.5:49544) 2023-06-23T01:04:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.243+0000 7f151d25e700 1 -- 172.21.15.5:0/1349803089 learned_addr learned my addr 172.21.15.5:0/1349803089 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151da5f700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1504004180 0x7f15181453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151e260700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15180ca670 0x7f15180cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151d25e700 1 -- 172.21.15.5:0/1349803089 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1504004180 msgr2=0x7f15181453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151d25e700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1504004180 0x7f15181453c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151d25e700 1 -- 172.21.15.5:0/1349803089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15180ca670 msgr2=0x7f15180cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151d25e700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15180ca670 0x7f15180cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151d25e700 1 -- 172.21.15.5:0/1349803089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151816a1b0 con 0x7f151816e130 2023-06-23T01:04:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151d25e700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f151816e130 0x7f15180af510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15140135b0 tx=0x7f151401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f150dffb700 1 -- 172.21.15.5:0/1349803089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514025070 con 0x7f151816e130 2023-06-23T01:04:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f150dffb700 1 -- 172.21.15.5:0/1349803089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514006af0 con 0x7f151816e130 2023-06-23T01:04:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f150dffb700 1 -- 172.21.15.5:0/1349803089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514006820 con 0x7f151816e130 2023-06-23T01:04:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151816a490 con 0x7f151816e130 2023-06-23T01:04:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.244+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151816aa50 con 0x7f151816e130 2023-06-23T01:04:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.245+0000 7f150dffb700 1 -- 172.21.15.5:0/1349803089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1514016450 con 0x7f151816e130 2023-06-23T01:04:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.245+0000 7f150dffb700 1 -- 172.21.15.5:0/1349803089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1514029760 con 0x7f151816e130 2023-06-23T01:04:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.246+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151804c1b0 con 0x7f151816e130 2023-06-23T01:04:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.248+0000 7f150dffb700 1 -- 172.21.15.5:0/1349803089 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f15140486e0 con 0x7f151816e130 2023-06-23T01:04:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.352+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f151804c1b0 con 0x7f151816e130 2023-06-23T01:04:29.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.355+0000 7f150dffb700 1 -- 172.21.15.5:0/1349803089 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1514037020 con 0x7f151816e130 2023-06-23T01:04:29.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f151816e130 msgr2=0x7f15180af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f151816e130 0x7f15180af510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f15140135b0 tx=0x7f151401e000 comp rx=0 tx=0).stop 2023-06-23T01:04:29.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 shutdown_connections 2023-06-23T01:04:29.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15180ca670 0x7f15180cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1504004180 0x7f15181453c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 --2- 172.21.15.5:0/1349803089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f151816e130 0x7f15180af510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 >> 172.21.15.5:0/1349803089 conn(0x7f151806e830 msgr2=0x7f15180cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:29.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 shutdown_connections 2023-06-23T01:04:29.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:29.356+0000 7f151fcc3700 1 -- 172.21.15.5:0/1349803089 wait complete. 2023-06-23T01:04:29.360 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2023-06-23T01:04:29.371 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-23T01:04:29.371 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:29.372 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-23T01:04:29.399 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:29.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-23T01:04:29.465 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:29.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-23T01:04:29.531 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-06-23T01:04:29.532 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-23T01:04:29.597 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:29.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-23T01:04:29.606 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:29.603+0000 7f9383017180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T01:04:29.623 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:29.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-23T01:04:29.687 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:29.687 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-23T01:04:29.698 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:29.695+0000 7fa30e5a8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T01:04:29.746 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:29.743+0000 7f9383017180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T01:04:29.752 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-06-23T01:04:29.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-23T01:04:29.817 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 2499edeb-6ba9-4392-b867-6a90f953982a 0 2023-06-23T01:04:29.825 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:29.822+0000 7f9383017180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T01:04:29.837 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:29.834+0000 7fa30e5a8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T01:04:29.916 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:29.913+0000 7fa30e5a8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T01:04:29.944 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.940+0000 7f9b7c177700 1 Processor -- start 2023-06-23T01:04:29.944 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.941+0000 7f9b7c177700 1 -- start start 2023-06-23T01:04:29.945 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.941+0000 7f9b7c177700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9b7415d4c0 0x7f9b740cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.946 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.941+0000 7f9b7c177700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b740ccbf0 con 0x7f9b7415dfa0 2023-06-23T01:04:29.946 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.941+0000 7f9b7c177700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b7406d420 con 0x7f9b740ae060 2023-06-23T01:04:29.946 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.941+0000 7f9b7c177700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b740cbfe0 con 0x7f9b7415d4c0 2023-06-23T01:04:29.946 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b79712700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9b740ae060 0x7f9b7415d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36160/0 (socket says 172.21.15.152:36160) 2023-06-23T01:04:29.946 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b79712700 1 -- 172.21.15.152:0/1025091469 learned_addr learned my addr 172.21.15.152:0/1025091469 (peer_addr_for_me v1:172.21.15.152:0/0) 2023-06-23T01:04:29.946 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b79f13700 1 --2- 172.21.15.152:0/1025091469 >> v2:172.21.15.5:3300/0 conn(0x7f9b7415d4c0 0x7f9b740cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.946 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b78f11700 1 -- 172.21.15.152:0/1025091469 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415436808 0 0) 0x7f9b7406d420 con 0x7f9b740ae060 2023-06-23T01:04:29.947 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b78f11700 1 -- 172.21.15.152:0/1025091469 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b60003570 con 0x7f9b740ae060 2023-06-23T01:04:29.947 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b78f11700 1 -- 172.21.15.152:0/1025091469 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 888674767 0 0) 0x7f9b60003570 con 0x7f9b740ae060 2023-06-23T01:04:29.947 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b79f13700 1 -- 172.21.15.152:0/1025091469 >> v1:172.21.15.152:6789/0 conn(0x7f9b740ae060 legacy=0x7f9b7415d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.947 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b79f13700 1 -- 172.21.15.152:0/1025091469 >> v1:172.21.15.5:6789/0 conn(0x7f9b7415dfa0 legacy=0x7f9b741591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.947 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b79f13700 1 -- 172.21.15.152:0/1025091469 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740cc2c0 con 0x7f9b7415d4c0 2023-06-23T01:04:29.947 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b79f13700 1 --2- 172.21.15.152:0/1025091469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b7415d4c0 0x7f9b740cb8a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009880 tx=0x7f9b64009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64305a5e3439e82f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:29.947 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.942+0000 7f9b78f11700 1 -- 172.21.15.152:0/1025091469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70001470 con 0x7f9b7415d4c0 2023-06-23T01:04:29.948 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b78f11700 1 -- 172.21.15.152:0/1025091469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b64011420 con 0x7f9b7415d4c0 2023-06-23T01:04:29.948 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b78f11700 1 -- 172.21.15.152:0/1025091469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b640117c0 con 0x7f9b7415d4c0 2023-06-23T01:04:29.950 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b7c177700 1 -- 172.21.15.152:0/1025091469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b7415d4c0 msgr2=0x7f9b740cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.950 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b7c177700 1 --2- 172.21.15.152:0/1025091469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b7415d4c0 0x7f9b740cb8a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009880 tx=0x7f9b64009120 comp rx=0 tx=0).stop 2023-06-23T01:04:29.950 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b7c177700 1 -- 172.21.15.152:0/1025091469 shutdown_connections 2023-06-23T01:04:29.950 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b7c177700 1 --2- 172.21.15.152:0/1025091469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b7415d4c0 0x7f9b740cb8a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.950 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b7c177700 1 -- 172.21.15.152:0/1025091469 >> 172.21.15.152:0/1025091469 conn(0x7f9b74097500 msgr2=0x7f9b74087fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:29.951 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b7c177700 1 -- 172.21.15.152:0/1025091469 shutdown_connections 2023-06-23T01:04:29.951 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.943+0000 7f9b7c177700 1 -- 172.21.15.152:0/1025091469 wait complete. 2023-06-23T01:04:29.951 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.944+0000 7f9b7c177700 1 Processor -- start 2023-06-23T01:04:29.951 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.944+0000 7f9b7c177700 1 -- start start 2023-06-23T01:04:29.951 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7c177700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b740ae060 0x7f9b74157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.951 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7c177700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b7415d4c0 0x7f9b740b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.951 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7c177700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b7415dfa0 0x7f9b7416cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:29.952 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7c177700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740b2c00 con 0x7f9b740ae060 2023-06-23T01:04:29.952 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7c177700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740b1ff0 con 0x7f9b7415dfa0 2023-06-23T01:04:29.952 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7c177700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740b22d0 con 0x7f9b7415d4c0 2023-06-23T01:04:29.952 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b79712700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b7415d4c0 0x7f9b740b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.952 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7a714700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b7415dfa0 0x7f9b7416cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.952 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b79712700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b7415d4c0 0x7f9b740b3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:38420/0 (socket says 172.21.15.152:38420) 2023-06-23T01:04:29.952 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b79712700 1 -- 172.21.15.152:0/1076200838 learned_addr learned my addr 172.21.15.152:0/1076200838 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:29.952 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b79f13700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b740ae060 0x7f9b74157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:29.953 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7a714700 1 -- 172.21.15.152:0/1076200838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b7415d4c0 msgr2=0x7f9b740b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.953 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7a714700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b7415d4c0 0x7f9b740b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.953 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7a714700 1 -- 172.21.15.152:0/1076200838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b740ae060 msgr2=0x7f9b74157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:29.953 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7a714700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b740ae060 0x7f9b74157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:29.953 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b7a714700 1 -- 172.21.15.152:0/1076200838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b640093c0 con 0x7f9b7415dfa0 2023-06-23T01:04:29.953 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.945+0000 7f9b79712700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b7415d4c0 0x7f9b740b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:29.953 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.946+0000 7f9b7a714700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b7415dfa0 0x7f9b7416cc60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c960 tx=0x7f9b6c0016d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:29.954 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.946+0000 7f9b69ffb700 1 -- 172.21.15.152:0/1076200838 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c012630 con 0x7f9b7415dfa0 2023-06-23T01:04:29.954 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.946+0000 7f9b69ffb700 1 -- 172.21.15.152:0/1076200838 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c0127b0 con 0x7f9b7415dfa0 2023-06-23T01:04:29.954 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.946+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b740b25b0 con 0x7f9b7415dfa0 2023-06-23T01:04:29.954 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.946+0000 7f9b69ffb700 1 -- 172.21.15.152:0/1076200838 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c012b70 con 0x7f9b7415dfa0 2023-06-23T01:04:29.954 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.946+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b741579b0 con 0x7f9b7415dfa0 2023-06-23T01:04:29.954 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.946+0000 7f9b69ffb700 1 -- 172.21.15.152:0/1076200838 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9b6c003030 con 0x7f9b7415dfa0 2023-06-23T01:04:29.954 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.946+0000 7f9b69ffb700 1 -- 172.21.15.152:0/1076200838 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9b6c01be10 con 0x7f9b7415dfa0 2023-06-23T01:04:29.954 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.947+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b740b2d70 con 0x7f9b7415dfa0 2023-06-23T01:04:29.955 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:29.949+0000 7f9b69ffb700 1 -- 172.21.15.152:0/1076200838 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f9b6c01b400 con 0x7f9b7415dfa0 2023-06-23T01:04:30.020 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:30.017+0000 7f9383017180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:04:30.034 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.031+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2499edeb-6ba9-4392-b867-6a90f953982a", "id": 0} v 0) v1 -- 0x7f9b7404c1b0 con 0x7f9b7415dfa0 2023-06-23T01:04:30.037 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.034+0000 7f9b69ffb700 1 -- 172.21.15.152:0/1076200838 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2499edeb-6ba9-4392-b867-6a90f953982a", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9b6c01b5a0 con 0x7f9b7415dfa0 2023-06-23T01:04:30.038 INFO:teuthology.orchestra.run.smithi152.stdout:0 2023-06-23T01:04:30.039 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.035+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b7415dfa0 msgr2=0x7f9b7416cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.039 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.035+0000 7f9b7c177700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b7415dfa0 0x7f9b7416cc60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c960 tx=0x7f9b6c0016d0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.039 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.035+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 shutdown_connections 2023-06-23T01:04:30.039 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.035+0000 7f9b7c177700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9b7415dfa0 0x7f9b7416cc60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.039 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.035+0000 7f9b7c177700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b740ae060 0x7f9b74157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.039 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.035+0000 7f9b7c177700 1 --2- 172.21.15.152:0/1076200838 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9b7415d4c0 0x7f9b740b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.040 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.035+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 >> 172.21.15.152:0/1076200838 conn(0x7f9b74097500 msgr2=0x7f9b74157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.040 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.036+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 shutdown_connections 2023-06-23T01:04:30.040 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.036+0000 7f9b7c177700 1 -- 172.21.15.152:0/1076200838 wait complete. 2023-06-23T01:04:30.050 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new f2aa5a85-0aea-4bef-b45a-78297c1cd686 1 2023-06-23T01:04:30.108 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:30.105+0000 7fa30e5a8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:04:30.166 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.163+0000 7f44793fb700 1 Processor -- start 2023-06-23T01:04:30.167 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.163+0000 7f44793fb700 1 -- start start 2023-06-23T01:04:30.169 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f44793fb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4474156ea0 0x7f4474157270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.169 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f44793fb700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f447415dc30 0x7f4474159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.169 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f44793fb700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447415d7a0 con 0x7f447415a190 2023-06-23T01:04:30.169 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f44793fb700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447415a780 con 0x7f447415dc30 2023-06-23T01:04:30.169 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f44793fb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447415aa60 con 0x7f4474156ea0 2023-06-23T01:04:30.169 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f4472ffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4474156ea0 0x7f4474157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.169 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f4472ffd700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4474156ea0 0x7f4474157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:38422/0 (socket says 172.21.15.152:38422) 2023-06-23T01:04:30.170 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f4472ffd700 1 -- 172.21.15.152:0/3434839636 learned_addr learned my addr 172.21.15.152:0/3434839636 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:30.170 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.164+0000 7f44727fc700 1 --2- 172.21.15.152:0/3434839636 >> v2:172.21.15.5:3300/0 conn(0x7f447415dc30 0x7f4474159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.170 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f4472ffd700 1 -- 172.21.15.152:0/3434839636 >> v2:172.21.15.5:3300/0 conn(0x7f447415dc30 msgr2=0x7f4474159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:30.170 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f4472ffd700 1 --2- 172.21.15.152:0/3434839636 >> v2:172.21.15.5:3300/0 conn(0x7f447415dc30 0x7f4474159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.170 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f4472ffd700 1 -- 172.21.15.152:0/3434839636 >> v1:172.21.15.5:6789/0 conn(0x7f447415a190 legacy=0x7f44740a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.170 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f4472ffd700 1 -- 172.21.15.152:0/3434839636 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447415d430 con 0x7f4474156ea0 2023-06-23T01:04:30.170 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f4472ffd700 1 --2- 172.21.15.152:0/3434839636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4474156ea0 0x7f4474157270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44680099b0 tx=0x7f4468009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8cac05a0b0d68f0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:30.170 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f4471ffb700 1 -- 172.21.15.152:0/3434839636 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468009ea0 con 0x7f4474156ea0 2023-06-23T01:04:30.171 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f4471ffb700 1 -- 172.21.15.152:0/3434839636 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446800b860 con 0x7f4474156ea0 2023-06-23T01:04:30.171 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f4471ffb700 1 -- 172.21.15.152:0/3434839636 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800bc00 con 0x7f4474156ea0 2023-06-23T01:04:30.171 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f44793fb700 1 -- 172.21.15.152:0/3434839636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4474156ea0 msgr2=0x7f4474157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.171 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f44793fb700 1 --2- 172.21.15.152:0/3434839636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4474156ea0 0x7f4474157270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44680099b0 tx=0x7f4468009250 comp rx=0 tx=0).stop 2023-06-23T01:04:30.171 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f44793fb700 1 -- 172.21.15.152:0/3434839636 shutdown_connections 2023-06-23T01:04:30.171 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f44793fb700 1 --2- 172.21.15.152:0/3434839636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4474156ea0 0x7f4474157270 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.171 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f44793fb700 1 --2- 172.21.15.152:0/3434839636 >> v2:172.21.15.5:3300/0 conn(0x7f447415dc30 0x7f4474159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.172 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.165+0000 7f44793fb700 1 -- 172.21.15.152:0/3434839636 >> 172.21.15.152:0/3434839636 conn(0x7f447406e830 msgr2=0x7f447406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.172 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.166+0000 7f44793fb700 1 -- 172.21.15.152:0/3434839636 shutdown_connections 2023-06-23T01:04:30.172 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.166+0000 7f44793fb700 1 -- 172.21.15.152:0/3434839636 wait complete. 2023-06-23T01:04:30.174 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.167+0000 7f44793fb700 1 Processor -- start 2023-06-23T01:04:30.174 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.167+0000 7f44793fb700 1 -- start start 2023-06-23T01:04:30.174 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.167+0000 7f44793fb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4474156ea0 0x7f447414a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.174 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.167+0000 7f44793fb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447415a190 0x7f44740a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.174 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.167+0000 7f44793fb700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f447415dc30 0x7f44740a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.174 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.167+0000 7f44793fb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740ac0c0 con 0x7f447415a190 2023-06-23T01:04:30.175 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.167+0000 7f44793fb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44740b11a0 con 0x7f4474156ea0 2023-06-23T01:04:30.175 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.167+0000 7f44793fb700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44740aba70 con 0x7f447415dc30 2023-06-23T01:04:30.175 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4474156ea0 0x7f447414a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.175 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4474156ea0 0x7f447414a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49084/0 (socket says 172.21.15.152:49084) 2023-06-23T01:04:30.175 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f44737fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f447415dc30 0x7f44740a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.175 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 -- 172.21.15.152:0/3498116613 learned_addr learned my addr 172.21.15.152:0/3498116613 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:30.175 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f44727fc700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447415a190 0x7f44740a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.175 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 -- 172.21.15.152:0/3498116613 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f447415dc30 msgr2=0x7f44740a9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.176 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f447415dc30 0x7f44740a9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.176 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 -- 172.21.15.152:0/3498116613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447415a190 msgr2=0x7f44740a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.176 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447415a190 0x7f44740a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.176 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 -- 172.21.15.152:0/3498116613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44680094f0 con 0x7f4474156ea0 2023-06-23T01:04:30.176 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f44737fe700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f447415dc30 0x7f44740a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:30.176 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f4472ffd700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4474156ea0 0x7f447414a6e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f447415aba0 tx=0x7f4468017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:30.176 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f445affd700 1 -- 172.21.15.152:0/3498116613 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468000d10 con 0x7f4474156ea0 2023-06-23T01:04:30.177 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f445affd700 1 -- 172.21.15.152:0/3498116613 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446800b9d0 con 0x7f4474156ea0 2023-06-23T01:04:30.177 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f445affd700 1 -- 172.21.15.152:0/3498116613 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468025420 con 0x7f4474156ea0 2023-06-23T01:04:30.177 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44740abd50 con 0x7f4474156ea0 2023-06-23T01:04:30.177 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447414ac20 con 0x7f4474156ea0 2023-06-23T01:04:30.177 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f445affd700 1 -- 172.21.15.152:0/3498116613 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4468023070 con 0x7f4474156ea0 2023-06-23T01:04:30.177 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.168+0000 7f445affd700 1 -- 172.21.15.152:0/3498116613 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f446802f890 con 0x7f4474156ea0 2023-06-23T01:04:30.177 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.169+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44740ab8d0 con 0x7f4474156ea0 2023-06-23T01:04:30.178 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.171+0000 7f445affd700 1 -- 172.21.15.152:0/3498116613 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f44740ab8d0 con 0x7f4474156ea0 2023-06-23T01:04:30.203 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:30.200+0000 7f9383017180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T01:04:30.256 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.253+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f2aa5a85-0aea-4bef-b45a-78297c1cd686", "id": 1} v 0) v1 -- 0x7f447404c1b0 con 0x7f4474156ea0 2023-06-23T01:04:30.260 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.257+0000 7f445affd700 1 -- 172.21.15.152:0/3498116613 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f2aa5a85-0aea-4bef-b45a-78297c1cd686", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f446802a070 con 0x7f4474156ea0 2023-06-23T01:04:30.260 INFO:teuthology.orchestra.run.smithi152.stdout:1 2023-06-23T01:04:30.261 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4474156ea0 msgr2=0x7f447414a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.262 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4474156ea0 0x7f447414a6e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f447415aba0 tx=0x7f4468017000 comp rx=0 tx=0).stop 2023-06-23T01:04:30.262 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 shutdown_connections 2023-06-23T01:04:30.262 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f447415dc30 0x7f44740a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.263 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4474156ea0 0x7f447414a6e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.263 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 --2- 172.21.15.152:0/3498116613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f447415a190 0x7f44740a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.263 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 >> 172.21.15.152:0/3498116613 conn(0x7f447406e830 msgr2=0x7f44740cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.263 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 shutdown_connections 2023-06-23T01:04:30.263 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.258+0000 7f44793fb700 1 -- 172.21.15.152:0/3498116613 wait complete. 2023-06-23T01:04:30.273 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 72aa25ff-c758-4780-9b74-fc1c316a3264 2 2023-06-23T01:04:30.284 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:30.281+0000 7f9383017180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T01:04:30.290 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:30.286+0000 7fa30e5a8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T01:04:30.370 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:30.367+0000 7fa30e5a8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T01:04:30.391 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.388+0000 7f97f7b07700 1 Processor -- start 2023-06-23T01:04:30.391 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.388+0000 7f97f7b07700 1 -- start start 2023-06-23T01:04:30.392 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f7b07700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f97f006f170 0x7f97f015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.392 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f7b07700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f015cf70 con 0x7f97f00cede0 2023-06-23T01:04:30.392 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f7b07700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f00cb470 con 0x7f97f00b2550 2023-06-23T01:04:30.392 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f7b07700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f00cb750 con 0x7f97f006f170 2023-06-23T01:04:30.392 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f60a4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f97f006f170 0x7f97f015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.393 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f60a4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f97f006f170 0x7f97f015d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49094/0 (socket says 172.21.15.152:49094) 2023-06-23T01:04:30.393 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f60a4700 1 -- 172.21.15.152:0/1057086649 learned_addr learned my addr 172.21.15.152:0/1057086649 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:30.394 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f60a4700 1 -- 172.21.15.152:0/1057086649 >> v1:172.21.15.152:6789/0 conn(0x7f97f00b2550 legacy=0x7f97f00ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.394 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f48a1700 1 -- 172.21.15.152:0/1057086649 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268440759 0 0) 0x7f97f015cf70 con 0x7f97f00cede0 2023-06-23T01:04:30.394 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f60a4700 1 -- 172.21.15.152:0/1057086649 >> v1:172.21.15.5:6789/0 conn(0x7f97f00cede0 legacy=0x7f97f006ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.395 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f60a4700 1 -- 172.21.15.152:0/1057086649 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f015d1c0 con 0x7f97f006f170 2023-06-23T01:04:30.395 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f60a4700 1 --2- 172.21.15.152:0/1057086649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f006f170 0x7f97f015d9c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009f20 tx=0x7f97e8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b8c9e8a8bc76381 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:30.395 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f48a1700 1 -- 172.21.15.152:0/1057086649 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e800e070 con 0x7f97f006f170 2023-06-23T01:04:30.395 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.389+0000 7f97f48a1700 1 -- 172.21.15.152:0/1057086649 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e800ba80 con 0x7f97f006f170 2023-06-23T01:04:30.395 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.390+0000 7f97f48a1700 1 -- 172.21.15.152:0/1057086649 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80092e0 con 0x7f97f006f170 2023-06-23T01:04:30.395 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.390+0000 7f97f7b07700 1 -- 172.21.15.152:0/1057086649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f006f170 msgr2=0x7f97f015d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.395 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.390+0000 7f97f7b07700 1 --2- 172.21.15.152:0/1057086649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f006f170 0x7f97f015d9c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009f20 tx=0x7f97e8009840 comp rx=0 tx=0).stop 2023-06-23T01:04:30.395 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.390+0000 7f97f7b07700 1 -- 172.21.15.152:0/1057086649 shutdown_connections 2023-06-23T01:04:30.396 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.390+0000 7f97f7b07700 1 --2- 172.21.15.152:0/1057086649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f006f170 0x7f97f015d9c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.396 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.390+0000 7f97f7b07700 1 -- 172.21.15.152:0/1057086649 >> 172.21.15.152:0/1057086649 conn(0x7f97f00d02a0 msgr2=0x7f97f00d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.396 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.390+0000 7f97f7b07700 1 -- 172.21.15.152:0/1057086649 shutdown_connections 2023-06-23T01:04:30.396 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.390+0000 7f97f7b07700 1 -- 172.21.15.152:0/1057086649 wait complete. 2023-06-23T01:04:30.398 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.391+0000 7f97f7b07700 1 Processor -- start 2023-06-23T01:04:30.398 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.391+0000 7f97f7b07700 1 -- start start 2023-06-23T01:04:30.398 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f7b07700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97f006f170 0x7f97f00c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.398 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f7b07700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97f00b2550 0x7f97f0145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.398 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f7b07700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f00cede0 0x7f97f015b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.399 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f7b07700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f015b9d0 con 0x7f97f00b2550 2023-06-23T01:04:30.399 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f7b07700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f00c5840 con 0x7f97f00cede0 2023-06-23T01:04:30.399 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f7b07700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f015c7e0 con 0x7f97f006f170 2023-06-23T01:04:30.399 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f60a4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f00cede0 0x7f97f015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.399 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f58a3700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97f006f170 0x7f97f00c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.399 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f60a4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f00cede0 0x7f97f015b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49106/0 (socket says 172.21.15.152:49106) 2023-06-23T01:04:30.399 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f60a4700 1 -- 172.21.15.152:0/3686103596 learned_addr learned my addr 172.21.15.152:0/3686103596 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:30.400 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f50a2700 1 --2- 172.21.15.152:0/3686103596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97f00b2550 0x7f97f0145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.400 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f60a4700 1 -- 172.21.15.152:0/3686103596 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97f006f170 msgr2=0x7f97f00c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.400 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f60a4700 1 --2- 172.21.15.152:0/3686103596 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97f006f170 0x7f97f00c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.400 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f60a4700 1 -- 172.21.15.152:0/3686103596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97f00b2550 msgr2=0x7f97f0145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.400 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f60a4700 1 --2- 172.21.15.152:0/3686103596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97f00b2550 0x7f97f0145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.400 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.392+0000 7f97f60a4700 1 -- 172.21.15.152:0/3686103596 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8009ae0 con 0x7f97f00cede0 2023-06-23T01:04:30.400 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.393+0000 7f97f60a4700 1 --2- 172.21.15.152:0/3686103596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f00cede0 0x7f97f015b490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97e8012000 tx=0x7f97e8018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:30.400 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.393+0000 7f97e5ffb700 1 -- 172.21.15.152:0/3686103596 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e8007d10 con 0x7f97f00cede0 2023-06-23T01:04:30.401 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.393+0000 7f97e5ffb700 1 -- 172.21.15.152:0/3686103596 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e8007e70 con 0x7f97f00cede0 2023-06-23T01:04:30.401 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.393+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f015bbd0 con 0x7f97f00cede0 2023-06-23T01:04:30.401 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.393+0000 7f97e5ffb700 1 -- 172.21.15.152:0/3686103596 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e801c640 con 0x7f97f00cede0 2023-06-23T01:04:30.401 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.393+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f015c190 con 0x7f97f00cede0 2023-06-23T01:04:30.401 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.393+0000 7f97e5ffb700 1 -- 172.21.15.152:0/3686103596 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f97e801c9b0 con 0x7f97f00cede0 2023-06-23T01:04:30.401 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.393+0000 7f97e5ffb700 1 -- 172.21.15.152:0/3686103596 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f97e8026a00 con 0x7f97f00cede0 2023-06-23T01:04:30.401 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.394+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f015c950 con 0x7f97f00cede0 2023-06-23T01:04:30.402 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.396+0000 7f97e5ffb700 1 -- 172.21.15.152:0/3686103596 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f97e802b060 con 0x7f97f00cede0 2023-06-23T01:04:30.428 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:30.425+0000 7f9383017180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T01:04:30.483 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.480+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72aa25ff-c758-4780-9b74-fc1c316a3264", "id": 2} v 0) v1 -- 0x7f97f004c1b0 con 0x7f97f00cede0 2023-06-23T01:04:30.487 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.483+0000 7f97e5ffb700 1 -- 172.21.15.152:0/3686103596 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72aa25ff-c758-4780-9b74-fc1c316a3264", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f97e8021070 con 0x7f97f00cede0 2023-06-23T01:04:30.488 INFO:teuthology.orchestra.run.smithi152.stdout:2 2023-06-23T01:04:30.489 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f00cede0 msgr2=0x7f97f015b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.489 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 --2- 172.21.15.152:0/3686103596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f00cede0 0x7f97f015b490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97e8012000 tx=0x7f97e8018000 comp rx=0 tx=0).stop 2023-06-23T01:04:30.489 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 shutdown_connections 2023-06-23T01:04:30.489 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 --2- 172.21.15.152:0/3686103596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f97f00cede0 0x7f97f015b490 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.489 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 --2- 172.21.15.152:0/3686103596 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f97f006f170 0x7f97f00c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.489 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 --2- 172.21.15.152:0/3686103596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97f00b2550 0x7f97f0145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.490 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 >> 172.21.15.152:0/3686103596 conn(0x7f97f00d02a0 msgr2=0x7f97f00d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.490 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 shutdown_connections 2023-06-23T01:04:30.490 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.484+0000 7f97f7b07700 1 -- 172.21.15.152:0/3686103596 wait complete. 2023-06-23T01:04:30.499 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 77dd6974-49b0-4288-b310-e7a0729ea2ae 3 2023-06-23T01:04:30.510 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:30.507+0000 7fa30e5a8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T01:04:30.527 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:30.524+0000 7f9383017180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T01:04:30.611 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:30.608+0000 7fa30e5a8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T01:04:30.643 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.640+0000 7fd0a1575700 1 Processor -- start 2023-06-23T01:04:30.643 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.640+0000 7fd0a1575700 1 -- start start 2023-06-23T01:04:30.645 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.640+0000 7fd0a1575700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd09c0cede0 0x7fd09c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.645 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.640+0000 7fd0a1575700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.645 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.640+0000 7fd0a1575700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09c15cf70 con 0x7fd09c0b2550 2023-06-23T01:04:30.645 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.640+0000 7fd0a1575700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09c0cb470 con 0x7fd09c0cede0 2023-06-23T01:04:30.646 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.640+0000 7fd0a1575700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0cb750 con 0x7fd09c0b2920 2023-06-23T01:04:30.646 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09a7fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd09c0cede0 0x7fd09c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.646 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09b7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.646 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09b7fe700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c06ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:38442/0 (socket says 172.21.15.152:38442) 2023-06-23T01:04:30.646 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09a7fc700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd09c0cede0 0x7fd09c15d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49118/0 (socket says 172.21.15.152:49118) 2023-06-23T01:04:30.646 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09b7fe700 1 -- 172.21.15.152:0/544065510 learned_addr learned my addr 172.21.15.152:0/544065510 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:30.646 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09b7fe700 1 -- 172.21.15.152:0/544065510 >> v2:172.21.15.152:3300/0 conn(0x7fd09c0cede0 msgr2=0x7fd09c15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:30.647 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09b7fe700 1 --2- 172.21.15.152:0/544065510 >> v2:172.21.15.152:3300/0 conn(0x7fd09c0cede0 0x7fd09c15d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.647 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09b7fe700 1 -- 172.21.15.152:0/544065510 >> v1:172.21.15.5:6789/0 conn(0x7fd09c0b2550 legacy=0x7fd09c0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.647 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09b7fe700 1 -- 172.21.15.152:0/544065510 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c15d1c0 con 0x7fd09c0b2920 2023-06-23T01:04:30.647 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09a7fc700 1 --2- 172.21.15.152:0/544065510 >> v2:172.21.15.152:3300/0 conn(0x7fd09c0cede0 0x7fd09c15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:30.647 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd09b7fe700 1 --2- 172.21.15.152:0/544065510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c06ea60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd094009f20 tx=0x7fd094009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ab21d7b3ccbf2d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:30.647 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd099ffb700 1 -- 172.21.15.152:0/544065510 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09400e070 con 0x7fd09c0b2920 2023-06-23T01:04:30.648 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd099ffb700 1 -- 172.21.15.152:0/544065510 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09400b9d0 con 0x7fd09c0b2920 2023-06-23T01:04:30.648 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.641+0000 7fd099ffb700 1 -- 172.21.15.152:0/544065510 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0940092e0 con 0x7fd09c0b2920 2023-06-23T01:04:30.648 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.642+0000 7fd0a1575700 1 -- 172.21.15.152:0/544065510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 msgr2=0x7fd09c06ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.648 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.642+0000 7fd0a1575700 1 --2- 172.21.15.152:0/544065510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c06ea60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd094009f20 tx=0x7fd094009840 comp rx=0 tx=0).stop 2023-06-23T01:04:30.648 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.642+0000 7fd0a1575700 1 -- 172.21.15.152:0/544065510 shutdown_connections 2023-06-23T01:04:30.648 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.642+0000 7fd0a1575700 1 --2- 172.21.15.152:0/544065510 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c06ea60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.648 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.642+0000 7fd0a1575700 1 --2- 172.21.15.152:0/544065510 >> v2:172.21.15.152:3300/0 conn(0x7fd09c0cede0 0x7fd09c15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.649 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.642+0000 7fd0a1575700 1 -- 172.21.15.152:0/544065510 >> 172.21.15.152:0/544065510 conn(0x7fd09c0d02a0 msgr2=0x7fd09c0d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.649 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.642+0000 7fd0a1575700 1 -- 172.21.15.152:0/544065510 shutdown_connections 2023-06-23T01:04:30.649 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.642+0000 7fd0a1575700 1 -- 172.21.15.152:0/544065510 wait complete. 2023-06-23T01:04:30.651 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.643+0000 7fd0a1575700 1 Processor -- start 2023-06-23T01:04:30.651 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.643+0000 7fd0a1575700 1 -- start start 2023-06-23T01:04:30.652 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd0a1575700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd09c0b2550 0x7fd09c150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.652 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd0a1575700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c0c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.652 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd0a1575700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd09c0cede0 0x7fd09c0c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.652 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd0a1575700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c16afe0 con 0x7fd09c0b2550 2023-06-23T01:04:30.652 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd0a1575700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c16a070 con 0x7fd09c0cede0 2023-06-23T01:04:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd0a1575700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c1514c0 con 0x7fd09c0b2920 2023-06-23T01:04:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c0c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c0c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:38452/0 (socket says 172.21.15.152:38452) 2023-06-23T01:04:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 -- 172.21.15.152:0/382911750 learned_addr learned my addr 172.21.15.152:0/382911750 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09b7fe700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd09c0cede0 0x7fd09c0c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.653 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09affd700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd09c0b2550 0x7fd09c150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 -- 172.21.15.152:0/382911750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd09c0cede0 msgr2=0x7fd09c0c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd09c0cede0 0x7fd09c0c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 -- 172.21.15.152:0/382911750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd09c0b2550 msgr2=0x7fd09c150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd09c0b2550 0x7fd09c150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 -- 172.21.15.152:0/382911750 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c009420 con 0x7fd09c0b2920 2023-06-23T01:04:30.654 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd09a7fc700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c0c61c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00ec80 tx=0x7fd08c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.644+0000 7fd082ffd700 1 -- 172.21.15.152:0/382911750 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00c850 con 0x7fd09c0b2920 2023-06-23T01:04:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.645+0000 7fd082ffd700 1 -- 172.21.15.152:0/382911750 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c00c9b0 con 0x7fd09c0b2920 2023-06-23T01:04:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.645+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd094009ae0 con 0x7fd09c0b2920 2023-06-23T01:04:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.645+0000 7fd082ffd700 1 -- 172.21.15.152:0/382911750 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00cd50 con 0x7fd09c0b2920 2023-06-23T01:04:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.645+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c16a6b0 con 0x7fd09c0b2920 2023-06-23T01:04:30.655 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.645+0000 7fd082ffd700 1 -- 172.21.15.152:0/382911750 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd08c005a00 con 0x7fd09c0b2920 2023-06-23T01:04:30.656 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.645+0000 7fd082ffd700 1 -- 172.21.15.152:0/382911750 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fd08c013da0 con 0x7fd09c0b2920 2023-06-23T01:04:30.656 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.646+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09c04c1b0 con 0x7fd09c0b2920 2023-06-23T01:04:30.656 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.648+0000 7fd082ffd700 1 -- 172.21.15.152:0/382911750 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fd08c021060 con 0x7fd09c0b2920 2023-06-23T01:04:30.732 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.729+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "77dd6974-49b0-4288-b310-e7a0729ea2ae", "id": 3} v 0) v1 -- 0x7fd09c16b120 con 0x7fd09c0b2920 2023-06-23T01:04:30.736 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.733+0000 7fd082ffd700 1 -- 172.21.15.152:0/382911750 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "77dd6974-49b0-4288-b310-e7a0729ea2ae", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd08c017070 con 0x7fd09c0b2920 2023-06-23T01:04:30.737 INFO:teuthology.orchestra.run.smithi152.stdout:3 2023-06-23T01:04:30.738 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 msgr2=0x7fd09c0c61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.739 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c0c61c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00ec80 tx=0x7fd08c0091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.739 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 shutdown_connections 2023-06-23T01:04:30.739 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd09c0cede0 0x7fd09c0c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.739 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd09c0b2550 0x7fd09c150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.740 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 --2- 172.21.15.152:0/382911750 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd09c0b2920 0x7fd09c0c61c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.740 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 >> 172.21.15.152:0/382911750 conn(0x7fd09c0d02a0 msgr2=0x7fd09c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.740 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 shutdown_connections 2023-06-23T01:04:30.741 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.734+0000 7fd0a1575700 1 -- 172.21.15.152:0/382911750 wait complete. 2023-06-23T01:04:30.749 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 80562fc5-676d-4cee-9978-4a94fca8df84 4 2023-06-23T01:04:30.868 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.864+0000 7f5f530d7700 1 Processor -- start 2023-06-23T01:04:30.868 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.865+0000 7f5f530d7700 1 -- start start 2023-06-23T01:04:30.869 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.865+0000 7f5f530d7700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f4c0ccf10 0x7f5f4c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.870 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.865+0000 7f5f530d7700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c0c7d00 con 0x7f5f4c155f60 2023-06-23T01:04:30.870 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.865+0000 7f5f530d7700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4c06ae90 con 0x7f5f4c0ad9e0 2023-06-23T01:04:30.870 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.865+0000 7f5f530d7700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4c0cd8f0 con 0x7f5f4c0ccf10 2023-06-23T01:04:30.870 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.865+0000 7f5f4bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f4c0ccf10 0x7f5f4c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.870 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4bfff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f4c0ccf10 0x7f5f4c0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49138/0 (socket says 172.21.15.152:49138) 2023-06-23T01:04:30.870 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4bfff700 1 -- 172.21.15.152:0/853281386 learned_addr learned my addr 172.21.15.152:0/853281386 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:30.870 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f51674700 1 --1- 172.21.15.152:0/853281386 >> v1:172.21.15.152:6789/0 conn(0x7f5f4c0ad9e0 0x7f5f4c0addf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36184/0 (socket says 172.21.15.152:36184) 2023-06-23T01:04:30.871 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4bfff700 1 -- 172.21.15.152:0/853281386 >> v1:172.21.15.152:6789/0 conn(0x7f5f4c0ad9e0 legacy=0x7f5f4c0addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.871 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4bfff700 1 -- 172.21.15.152:0/853281386 >> v1:172.21.15.5:6789/0 conn(0x7f5f4c155f60 legacy=0x7f5f4c06a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.871 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4b7fe700 1 -- 172.21.15.152:0/853281386 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730414302 0 0) 0x7f5f4c06ae90 con 0x7f5f4c0ad9e0 2023-06-23T01:04:30.871 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4bfff700 1 -- 172.21.15.152:0/853281386 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c06b170 con 0x7f5f4c0ccf10 2023-06-23T01:04:30.871 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4bfff700 1 --2- 172.21.15.152:0/853281386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c0ccf10 0x7f5f4c0cd3b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5f380099b0 tx=0x7f5f38009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=392264389d3ae162 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:30.871 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4b7fe700 1 -- 172.21.15.152:0/853281386 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800b860 con 0x7f5f4c0ccf10 2023-06-23T01:04:30.871 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4b7fe700 1 -- 172.21.15.152:0/853281386 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3800b9c0 con 0x7f5f4c0ccf10 2023-06-23T01:04:30.872 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.866+0000 7f5f4b7fe700 1 -- 172.21.15.152:0/853281386 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800bd60 con 0x7f5f4c0ccf10 2023-06-23T01:04:30.874 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.867+0000 7f5f530d7700 1 -- 172.21.15.152:0/853281386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c0ccf10 msgr2=0x7f5f4c0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.874 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.867+0000 7f5f530d7700 1 --2- 172.21.15.152:0/853281386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c0ccf10 0x7f5f4c0cd3b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5f380099b0 tx=0x7f5f38009250 comp rx=0 tx=0).stop 2023-06-23T01:04:30.874 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.867+0000 7f5f530d7700 1 -- 172.21.15.152:0/853281386 shutdown_connections 2023-06-23T01:04:30.875 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.867+0000 7f5f530d7700 1 --2- 172.21.15.152:0/853281386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c0ccf10 0x7f5f4c0cd3b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.875 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.867+0000 7f5f530d7700 1 -- 172.21.15.152:0/853281386 >> 172.21.15.152:0/853281386 conn(0x7f5f4c068240 msgr2=0x7f5f4c068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.875 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.867+0000 7f5f530d7700 1 -- 172.21.15.152:0/853281386 shutdown_connections 2023-06-23T01:04:30.875 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.867+0000 7f5f530d7700 1 -- 172.21.15.152:0/853281386 wait complete. 2023-06-23T01:04:30.875 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.868+0000 7f5f530d7700 1 Processor -- start 2023-06-23T01:04:30.875 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.868+0000 7f5f530d7700 1 -- start start 2023-06-23T01:04:30.875 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f530d7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f4c0ad9e0 0x7f5f4c150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.876 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f530d7700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f4c0ccf10 0x7f5f4c166710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.876 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f530d7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c155f60 0x7f5f4c0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:30.876 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f530d7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0c66b0 con 0x7f5f4c0ad9e0 2023-06-23T01:04:30.876 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f530d7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1505d0 con 0x7f5f4c155f60 2023-06-23T01:04:30.876 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f530d7700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c0c4960 con 0x7f5f4c0ccf10 2023-06-23T01:04:30.876 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f4bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f4c0ccf10 0x7f5f4c166710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.876 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f51674700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c155f60 0x7f5f4c0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.877 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f4bfff700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f4c0ccf10 0x7f5f4c166710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:38466/0 (socket says 172.21.15.152:38466) 2023-06-23T01:04:30.877 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f4bfff700 1 -- 172.21.15.152:0/2175572743 learned_addr learned my addr 172.21.15.152:0/2175572743 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:30.877 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f50e73700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f4c0ad9e0 0x7f5f4c150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:30.877 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f51674700 1 -- 172.21.15.152:0/2175572743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f4c0ccf10 msgr2=0x7f5f4c166710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.877 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f51674700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f4c0ccf10 0x7f5f4c166710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.877 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f51674700 1 -- 172.21.15.152:0/2175572743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f4c0ad9e0 msgr2=0x7f5f4c150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.877 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f51674700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f4c0ad9e0 0x7f5f4c150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.878 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f51674700 1 -- 172.21.15.152:0/2175572743 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380094f0 con 0x7f5f4c155f60 2023-06-23T01:04:30.878 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f4bfff700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f4c0ccf10 0x7f5f4c166710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:30.878 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.869+0000 7f5f51674700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c155f60 0x7f5f4c0c59c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400ef30 tx=0x7f5f44001af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:30.878 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.870+0000 7f5f48ff9700 1 -- 172.21.15.152:0/2175572743 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44005df0 con 0x7f5f4c155f60 2023-06-23T01:04:30.878 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.870+0000 7f5f48ff9700 1 -- 172.21.15.152:0/2175572743 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4400cb90 con 0x7f5f4c155f60 2023-06-23T01:04:30.878 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.870+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c166c50 con 0x7f5f4c155f60 2023-06-23T01:04:30.878 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.870+0000 7f5f48ff9700 1 -- 172.21.15.152:0/2175572743 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440070f0 con 0x7f5f4c155f60 2023-06-23T01:04:30.878 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.870+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c0c4310 con 0x7f5f4c155f60 2023-06-23T01:04:30.879 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.870+0000 7f5f48ff9700 1 -- 172.21.15.152:0/2175572743 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5f440074e0 con 0x7f5f4c155f60 2023-06-23T01:04:30.879 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.870+0000 7f5f48ff9700 1 -- 172.21.15.152:0/2175572743 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f5f44011ac0 con 0x7f5f4c155f60 2023-06-23T01:04:30.879 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.871+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c04c1b0 con 0x7f5f4c155f60 2023-06-23T01:04:30.879 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.873+0000 7f5f48ff9700 1 -- 172.21.15.152:0/2175572743 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f5f440396e0 con 0x7f5f4c155f60 2023-06-23T01:04:30.958 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.955+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "80562fc5-676d-4cee-9978-4a94fca8df84", "id": 4} v 0) v1 -- 0x7f5f4c0c4aa0 con 0x7f5f4c155f60 2023-06-23T01:04:30.961 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.958+0000 7f5f48ff9700 1 -- 172.21.15.152:0/2175572743 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80562fc5-676d-4cee-9978-4a94fca8df84", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f44028030 con 0x7f5f4c155f60 2023-06-23T01:04:30.961 INFO:teuthology.orchestra.run.smithi152.stdout:4 2023-06-23T01:04:30.962 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c155f60 msgr2=0x7f5f4c0c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:30.962 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c155f60 0x7f5f4c0c59c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400ef30 tx=0x7f5f44001af0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.962 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 shutdown_connections 2023-06-23T01:04:30.963 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f4c155f60 0x7f5f4c0c59c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.963 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f4c0ad9e0 0x7f5f4c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.963 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 --2- 172.21.15.152:0/2175572743 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f4c0ccf10 0x7f5f4c166710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:30.963 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 >> 172.21.15.152:0/2175572743 conn(0x7f5f4c068240 msgr2=0x7f5f4c0cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:30.963 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 shutdown_connections 2023-06-23T01:04:30.963 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:30.959+0000 7f5f530d7700 1 -- 172.21.15.152:0/2175572743 wait complete. 2023-06-23T01:04:30.974 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 1db79ad4-997e-4516-a40b-90ee44d4957a 5 2023-06-23T01:04:31.076 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:31.073+0000 7f9383017180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T01:04:31.090 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.087+0000 7f61eb976700 1 Processor -- start 2023-06-23T01:04:31.090 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.087+0000 7f61eb976700 1 -- start start 2023-06-23T01:04:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61eb976700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f61e40cede0 0x7f61e415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61eb976700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40b2920 0x7f61e406ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61eb976700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e415cf70 con 0x7f61e40b2550 2023-06-23T01:04:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61eb976700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e40cb470 con 0x7f61e40cede0 2023-06-23T01:04:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61eb976700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40cb750 con 0x7f61e40b2920 2023-06-23T01:04:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e9712700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f61e40b2550 0x7f61e40ca8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36190/0 (socket says 172.21.15.152:36190) 2023-06-23T01:04:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e9712700 1 -- 172.21.15.152:0/2043601226 learned_addr learned my addr 172.21.15.152:0/2043601226 (peer_addr_for_me v1:172.21.15.152:0/0) 2023-06-23T01:04:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e9f13700 1 --2- 172.21.15.152:0/2043601226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40b2920 0x7f61e406ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e8f11700 1 --2- 172.21.15.152:0/2043601226 >> v2:172.21.15.5:3300/0 conn(0x7f61e40cede0 0x7f61e415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61dbfff700 1 -- 172.21.15.152:0/2043601226 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902157487 0 0) 0x7f61e415cf70 con 0x7f61e40b2550 2023-06-23T01:04:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61dbfff700 1 -- 172.21.15.152:0/2043601226 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d0003570 con 0x7f61e40b2550 2023-06-23T01:04:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e9f13700 1 -- 172.21.15.152:0/2043601226 >> v2:172.21.15.5:3300/0 conn(0x7f61e40cede0 msgr2=0x7f61e415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:31.096 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e9f13700 1 --2- 172.21.15.152:0/2043601226 >> v2:172.21.15.5:3300/0 conn(0x7f61e40cede0 0x7f61e415d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.096 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e9f13700 1 -- 172.21.15.152:0/2043601226 >> v1:172.21.15.152:6789/0 conn(0x7f61e40b2550 legacy=0x7f61e40ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.096 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e9f13700 1 -- 172.21.15.152:0/2043601226 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e415d1c0 con 0x7f61e40b2920 2023-06-23T01:04:31.096 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.088+0000 7f61e9f13700 1 --2- 172.21.15.152:0/2043601226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40b2920 0x7f61e406ea60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009e60 tx=0x7f61dc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459a02056d8eb415 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:31.096 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61dbfff700 1 -- 172.21.15.152:0/2043601226 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00e070 con 0x7f61e40b2920 2023-06-23T01:04:31.096 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61dbfff700 1 -- 172.21.15.152:0/2043601226 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc00bab0 con 0x7f61e40b2920 2023-06-23T01:04:31.097 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61dbfff700 1 -- 172.21.15.152:0/2043601226 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00be70 con 0x7f61e40b2920 2023-06-23T01:04:31.097 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61eb976700 1 -- 172.21.15.152:0/2043601226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40b2920 msgr2=0x7f61e406ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.097 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61eb976700 1 --2- 172.21.15.152:0/2043601226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40b2920 0x7f61e406ea60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009e60 tx=0x7f61dc009780 comp rx=0 tx=0).stop 2023-06-23T01:04:31.097 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61eb976700 1 -- 172.21.15.152:0/2043601226 shutdown_connections 2023-06-23T01:04:31.097 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61eb976700 1 --2- 172.21.15.152:0/2043601226 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40b2920 0x7f61e406ea60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.097 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61eb976700 1 --2- 172.21.15.152:0/2043601226 >> v2:172.21.15.5:3300/0 conn(0x7f61e40cede0 0x7f61e415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61eb976700 1 -- 172.21.15.152:0/2043601226 >> 172.21.15.152:0/2043601226 conn(0x7f61e40d02a0 msgr2=0x7f61e40d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61eb976700 1 -- 172.21.15.152:0/2043601226 shutdown_connections 2023-06-23T01:04:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.089+0000 7f61eb976700 1 -- 172.21.15.152:0/2043601226 wait complete. 2023-06-23T01:04:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.090+0000 7f61eb976700 1 Processor -- start 2023-06-23T01:04:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.090+0000 7f61eb976700 1 -- start start 2023-06-23T01:04:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61eb976700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e40b2550 0x7f61e40c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61eb976700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61e40b2920 0x7f61e414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61eb976700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40cede0 0x7f61e41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61eb976700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4155bb0 con 0x7f61e40b2550 2023-06-23T01:04:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61eb976700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e414b5e0 con 0x7f61e40b2920 2023-06-23T01:04:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61eb976700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e40c6070 con 0x7f61e40cede0 2023-06-23T01:04:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61e8f11700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61e40b2920 0x7f61e414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61e8f11700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61e40b2920 0x7f61e414b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49152/0 (socket says 172.21.15.152:49152) 2023-06-23T01:04:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61e8f11700 1 -- 172.21.15.152:0/1118103809 learned_addr learned my addr 172.21.15.152:0/1118103809 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.091+0000 7f61e9f13700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40cede0 0x7f61e41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61e9712700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e40b2550 0x7f61e40c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61e9f13700 1 -- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61e40b2920 msgr2=0x7f61e414b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61e9f13700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61e40b2920 0x7f61e414b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61e9f13700 1 -- 172.21.15.152:0/1118103809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e40b2550 msgr2=0x7f61e40c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61e9f13700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e40b2550 0x7f61e40c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61e9f13700 1 -- 172.21.15.152:0/1118103809 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc009a20 con 0x7f61e40cede0 2023-06-23T01:04:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61e8f11700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61e40b2920 0x7f61e414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61e9f13700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40cede0 0x7f61e41524f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f61dc016000 tx=0x7f61dc019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61d97fa700 1 -- 172.21.15.152:0/1118103809 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc012e50 con 0x7f61e40cede0 2023-06-23T01:04:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61d97fa700 1 -- 172.21.15.152:0/1118103809 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc027470 con 0x7f61e40cede0 2023-06-23T01:04:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e40c6350 con 0x7f61e40cede0 2023-06-23T01:04:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61d97fa700 1 -- 172.21.15.152:0/1118103809 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc027830 con 0x7f61e40cede0 2023-06-23T01:04:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e40c6d80 con 0x7f61e40cede0 2023-06-23T01:04:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61d97fa700 1 -- 172.21.15.152:0/1118103809 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f61dc024070 con 0x7f61e40cede0 2023-06-23T01:04:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.092+0000 7f61d97fa700 1 -- 172.21.15.152:0/1118103809 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f61dc01dae0 con 0x7f61e40cede0 2023-06-23T01:04:31.111 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.093+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e4155d20 con 0x7f61e40cede0 2023-06-23T01:04:31.111 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.095+0000 7f61d97fa700 1 -- 172.21.15.152:0/1118103809 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f61e4155d20 con 0x7f61e40cede0 2023-06-23T01:04:31.140 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:31.137+0000 7fa30e5a8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T01:04:31.157 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:31.154+0000 7f9383017180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T01:04:31.180 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.177+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1db79ad4-997e-4516-a40b-90ee44d4957a", "id": 5} v 0) v1 -- 0x7f61e404c1b0 con 0x7f61e40cede0 2023-06-23T01:04:31.186 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.183+0000 7f61d97fa700 1 -- 172.21.15.152:0/1118103809 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1db79ad4-997e-4516-a40b-90ee44d4957a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f61dc01dd40 con 0x7f61e40cede0 2023-06-23T01:04:31.186 INFO:teuthology.orchestra.run.smithi152.stdout:5 2023-06-23T01:04:31.189 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40cede0 msgr2=0x7f61e41524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.189 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40cede0 0x7f61e41524f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f61dc016000 tx=0x7f61dc019000 comp rx=0 tx=0).stop 2023-06-23T01:04:31.190 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 shutdown_connections 2023-06-23T01:04:31.190 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f61e40cede0 0x7f61e41524f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.190 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e40b2550 0x7f61e40c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.190 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 --2- 172.21.15.152:0/1118103809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f61e40b2920 0x7f61e414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.190 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 >> 172.21.15.152:0/1118103809 conn(0x7f61e40d02a0 msgr2=0x7f61e40ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 shutdown_connections 2023-06-23T01:04:31.191 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.184+0000 7f61eb976700 1 -- 172.21.15.152:0/1118103809 wait complete. 2023-06-23T01:04:31.199 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new c18d6095-c8f0-4d76-94a8-f8de40a45195 6 2023-06-23T01:04:31.222 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:31.218+0000 7fa30e5a8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T01:04:31.240 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:31.237+0000 7f9383017180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T01:04:31.304 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:31.300+0000 7fa30e5a8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T01:04:31.316 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.312+0000 7efd25416700 1 Processor -- start 2023-06-23T01:04:31.316 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.312+0000 7efd25416700 1 -- start start 2023-06-23T01:04:31.318 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd25416700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200cede0 0x7efd2015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.318 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd25416700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efd200b2920 0x7efd2006ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.319 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd25416700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2015cf70 con 0x7efd200b2550 2023-06-23T01:04:31.319 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd25416700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd200cb470 con 0x7efd200b2920 2023-06-23T01:04:31.319 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd25416700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd200cb750 con 0x7efd200cede0 2023-06-23T01:04:31.319 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd1f7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efd200b2920 0x7efd2006ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.320 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd1e7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200cede0 0x7efd2015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.320 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd1f7fe700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efd200b2920 0x7efd2006ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49162/0 (socket says 172.21.15.152:49162) 2023-06-23T01:04:31.320 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd1f7fe700 1 -- 172.21.15.152:0/1080541816 learned_addr learned my addr 172.21.15.152:0/1080541816 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:31.321 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.313+0000 7efd1effd700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7efd200b2550 0x7efd200ca8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36200/0 (socket says 172.21.15.152:36200) 2023-06-23T01:04:31.321 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1f7fe700 1 -- 172.21.15.152:0/1080541816 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200cede0 msgr2=0x7efd2015d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:31.321 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1f7fe700 1 --2- 172.21.15.152:0/1080541816 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200cede0 0x7efd2015d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.321 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1f7fe700 1 -- 172.21.15.152:0/1080541816 >> v1:172.21.15.152:6789/0 conn(0x7efd200b2550 legacy=0x7efd200ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.322 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1f7fe700 1 -- 172.21.15.152:0/1080541816 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2015d1c0 con 0x7efd200b2920 2023-06-23T01:04:31.322 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1dffb700 1 -- 172.21.15.152:0/1080541816 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329656116 0 0) 0x7efd2015cf70 con 0x7efd200b2550 2023-06-23T01:04:31.322 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1e7fc700 1 --2- 172.21.15.152:0/1080541816 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200cede0 0x7efd2015d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:31.322 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1f7fe700 1 --2- 172.21.15.152:0/1080541816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200b2920 0x7efd2006ea60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd18009f20 tx=0x7efd18009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0c054ec03167bf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:31.323 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1dffb700 1 -- 172.21.15.152:0/1080541816 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1800e070 con 0x7efd200b2920 2023-06-23T01:04:31.323 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1dffb700 1 -- 172.21.15.152:0/1080541816 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1800b9d0 con 0x7efd200b2920 2023-06-23T01:04:31.323 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd1dffb700 1 -- 172.21.15.152:0/1080541816 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd180092e0 con 0x7efd200b2920 2023-06-23T01:04:31.323 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd25416700 1 -- 172.21.15.152:0/1080541816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200b2920 msgr2=0x7efd2006ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.324 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd25416700 1 --2- 172.21.15.152:0/1080541816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200b2920 0x7efd2006ea60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efd18009f20 tx=0x7efd18009840 comp rx=0 tx=0).stop 2023-06-23T01:04:31.324 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd25416700 1 -- 172.21.15.152:0/1080541816 shutdown_connections 2023-06-23T01:04:31.324 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.314+0000 7efd25416700 1 --2- 172.21.15.152:0/1080541816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200b2920 0x7efd2006ea60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.324 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.315+0000 7efd25416700 1 --2- 172.21.15.152:0/1080541816 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200cede0 0x7efd2015d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.325 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.315+0000 7efd25416700 1 -- 172.21.15.152:0/1080541816 >> 172.21.15.152:0/1080541816 conn(0x7efd200d02a0 msgr2=0x7efd200d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:31.325 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.315+0000 7efd25416700 1 -- 172.21.15.152:0/1080541816 shutdown_connections 2023-06-23T01:04:31.325 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.315+0000 7efd25416700 1 -- 172.21.15.152:0/1080541816 wait complete. 2023-06-23T01:04:31.327 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.316+0000 7efd25416700 1 Processor -- start 2023-06-23T01:04:31.327 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.316+0000 7efd25416700 1 -- start start 2023-06-23T01:04:31.327 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.316+0000 7efd25416700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd200b2550 0x7efd200b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.316+0000 7efd25416700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200b2920 0x7efd201551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.316+0000 7efd25416700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200cede0 0x7efd200b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.316+0000 7efd25416700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd200b3ed0 con 0x7efd200b2550 2023-06-23T01:04:31.328 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd25416700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd200b0920 con 0x7efd200cede0 2023-06-23T01:04:31.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd25416700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd200b4e80 con 0x7efd200b2920 2023-06-23T01:04:31.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200b2920 0x7efd201551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200b2920 0x7efd201551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:38506/0 (socket says 172.21.15.152:38506) 2023-06-23T01:04:31.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1f7fe700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200cede0 0x7efd200b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 -- 172.21.15.152:0/3152147559 learned_addr learned my addr 172.21.15.152:0/3152147559 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:31.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1effd700 1 --2- 172.21.15.152:0/3152147559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd200b2550 0x7efd200b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.329 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 -- 172.21.15.152:0/3152147559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200cede0 msgr2=0x7efd200b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 --2- 172.21.15.152:0/3152147559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200cede0 0x7efd200b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 -- 172.21.15.152:0/3152147559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd200b2550 msgr2=0x7efd200b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 --2- 172.21.15.152:0/3152147559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd200b2550 0x7efd200b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 -- 172.21.15.152:0/3152147559 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd10009420 con 0x7efd200b2920 2023-06-23T01:04:31.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd1e7fc700 1 --2- 172.21.15.152:0/3152147559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200b2920 0x7efd201551d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd1000ec80 tx=0x7efd100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:31.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd06ffd700 1 -- 172.21.15.152:0/3152147559 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000c850 con 0x7efd200b2920 2023-06-23T01:04:31.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd06ffd700 1 -- 172.21.15.152:0/3152147559 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1000c9b0 con 0x7efd200b2920 2023-06-23T01:04:31.330 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd18009ae0 con 0x7efd200b2920 2023-06-23T01:04:31.331 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd06ffd700 1 -- 172.21.15.152:0/3152147559 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000cd30 con 0x7efd200b2920 2023-06-23T01:04:31.331 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.317+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd200b4420 con 0x7efd200b2920 2023-06-23T01:04:31.331 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.318+0000 7efd06ffd700 1 -- 172.21.15.152:0/3152147559 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7efd10005c80 con 0x7efd200b2920 2023-06-23T01:04:31.331 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.318+0000 7efd06ffd700 1 -- 172.21.15.152:0/3152147559 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7efd10013bd0 con 0x7efd200b2920 2023-06-23T01:04:31.331 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.318+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd200b47a0 con 0x7efd200b2920 2023-06-23T01:04:31.331 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.320+0000 7efd06ffd700 1 -- 172.21.15.152:0/3152147559 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7efd10021060 con 0x7efd200b2920 2023-06-23T01:04:31.404 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.401+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c18d6095-c8f0-4d76-94a8-f8de40a45195", "id": 6} v 0) v1 -- 0x7efd2004c1b0 con 0x7efd200b2920 2023-06-23T01:04:31.412 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.409+0000 7efd06ffd700 1 -- 172.21.15.152:0/3152147559 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c18d6095-c8f0-4d76-94a8-f8de40a45195", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7efd10017070 con 0x7efd200b2920 2023-06-23T01:04:31.412 INFO:teuthology.orchestra.run.smithi152.stdout:6 2023-06-23T01:04:31.414 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200b2920 msgr2=0x7efd201551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.415 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 --2- 172.21.15.152:0/3152147559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200b2920 0x7efd201551d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd1000ec80 tx=0x7efd100091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.415 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 shutdown_connections 2023-06-23T01:04:31.415 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 --2- 172.21.15.152:0/3152147559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efd200cede0 0x7efd200b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.415 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 --2- 172.21.15.152:0/3152147559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd200b2550 0x7efd200b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.416 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 --2- 172.21.15.152:0/3152147559 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efd200b2920 0x7efd201551d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.416 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 >> 172.21.15.152:0/3152147559 conn(0x7efd200d02a0 msgr2=0x7efd201569b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:31.416 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 shutdown_connections 2023-06-23T01:04:31.417 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.410+0000 7efd25416700 1 -- 172.21.15.152:0/3152147559 wait complete. 2023-06-23T01:04:31.426 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 6ade832e-0e46-4c4f-8fb2-3a86a643e19b 7 2023-06-23T01:04:31.581 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.577+0000 7f4b40d4a700 1 Processor -- start 2023-06-23T01:04:31.581 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.577+0000 7f4b40d4a700 1 -- start start 2023-06-23T01:04:31.583 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.578+0000 7f4b40d4a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.584 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.578+0000 7f4b40d4a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3c15cf70 con 0x7f4b3c0cede0 2023-06-23T01:04:31.584 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.578+0000 7f4b40d4a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3c0cb470 con 0x7f4b3c0b2550 2023-06-23T01:04:31.585 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.578+0000 7f4b40d4a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0cb750 con 0x7f4b3c06f170 2023-06-23T01:04:31.585 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3ad9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.585 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3ad9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c15d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:38512/0 (socket says 172.21.15.152:38512) 2023-06-23T01:04:31.585 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3ad9d700 1 -- 172.21.15.152:0/3491677401 learned_addr learned my addr 172.21.15.152:0/3491677401 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:31.588 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3a59c700 1 --1- 172.21.15.152:0/3491677401 >> v1:172.21.15.152:6789/0 conn(0x7f4b3c0b2550 0x7f4b3c0ca8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:36202/0 (socket says 172.21.15.152:36202) 2023-06-23T01:04:31.588 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3ad9d700 1 -- 172.21.15.152:0/3491677401 >> v1:172.21.15.152:6789/0 conn(0x7f4b3c0b2550 legacy=0x7f4b3c0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.589 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3ad9d700 1 -- 172.21.15.152:0/3491677401 >> v1:172.21.15.5:6789/0 conn(0x7f4b3c0cede0 legacy=0x7f4b3c06ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.589 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3959a700 1 -- 172.21.15.152:0/3491677401 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390316880 0 0) 0x7f4b3c0cb470 con 0x7f4b3c0b2550 2023-06-23T01:04:31.589 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3ad9d700 1 -- 172.21.15.152:0/3491677401 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c15d1c0 con 0x7f4b3c06f170 2023-06-23T01:04:31.589 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3ad9d700 1 --2- 172.21.15.152:0/3491677401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c15d9c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009f50 tx=0x7f4b34009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1975fb84bbd01f37 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:31.589 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3959a700 1 -- 172.21.15.152:0/3491677401 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3400e070 con 0x7f4b3c06f170 2023-06-23T01:04:31.589 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3959a700 1 -- 172.21.15.152:0/3491677401 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3400bab0 con 0x7f4b3c06f170 2023-06-23T01:04:31.589 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.579+0000 7f4b3959a700 1 -- 172.21.15.152:0/3491677401 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b340092e0 con 0x7f4b3c06f170 2023-06-23T01:04:31.590 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.580+0000 7f4b40d4a700 1 -- 172.21.15.152:0/3491677401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 msgr2=0x7f4b3c15d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.590 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.580+0000 7f4b40d4a700 1 --2- 172.21.15.152:0/3491677401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c15d9c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009f50 tx=0x7f4b34009840 comp rx=0 tx=0).stop 2023-06-23T01:04:31.590 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.580+0000 7f4b40d4a700 1 -- 172.21.15.152:0/3491677401 shutdown_connections 2023-06-23T01:04:31.590 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.580+0000 7f4b40d4a700 1 --2- 172.21.15.152:0/3491677401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c15d9c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.590 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.580+0000 7f4b40d4a700 1 -- 172.21.15.152:0/3491677401 >> 172.21.15.152:0/3491677401 conn(0x7f4b3c0d02a0 msgr2=0x7f4b3c0d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:31.590 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.580+0000 7f4b40d4a700 1 -- 172.21.15.152:0/3491677401 shutdown_connections 2023-06-23T01:04:31.590 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.580+0000 7f4b40d4a700 1 -- 172.21.15.152:0/3491677401 wait complete. 2023-06-23T01:04:31.592 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.581+0000 7f4b40d4a700 1 Processor -- start 2023-06-23T01:04:31.596 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.581+0000 7f4b40d4a700 1 -- start start 2023-06-23T01:04:31.597 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b40d4a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.597 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b40d4a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c0b2550 0x7f4b3c145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.597 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b40d4a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b3c0cede0 0x7f4b3c0b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.597 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b40d4a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c15bf30 con 0x7f4b3c0b2550 2023-06-23T01:04:31.597 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b40d4a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0ca170 con 0x7f4b3c0cede0 2023-06-23T01:04:31.597 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b40d4a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c15b320 con 0x7f4b3c06f170 2023-06-23T01:04:31.597 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3ad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b3c0cede0 0x7f4b3c0b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.598 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3ad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b3c0cede0 0x7f4b3c0b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49186/0 (socket says 172.21.15.152:49186) 2023-06-23T01:04:31.598 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3ad9d700 1 -- 172.21.15.152:0/2255049616 learned_addr learned my addr 172.21.15.152:0/2255049616 (peer_addr_for_me v2:172.21.15.152:0/0) 2023-06-23T01:04:31.598 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3a59c700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.598 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b39d9b700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c0b2550 0x7f4b3c145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.598 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3a59c700 1 -- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b3c0cede0 msgr2=0x7f4b3c0b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.598 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3a59c700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b3c0cede0 0x7f4b3c0b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.598 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3a59c700 1 -- 172.21.15.152:0/2255049616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c0b2550 msgr2=0x7f4b3c145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.598 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3a59c700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c0b2550 0x7f4b3c145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.599 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3a59c700 1 -- 172.21.15.152:0/2255049616 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b34009ae0 con 0x7f4b3c06f170 2023-06-23T01:04:31.599 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3ad9d700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b3c0cede0 0x7f4b3c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:31.599 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.582+0000 7f4b3a59c700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c0c9c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4b300011e0 tx=0x7f4b30007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:31.599 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.583+0000 7f4b2a7fc700 1 -- 172.21.15.152:0/2255049616 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30005a70 con 0x7f4b3c06f170 2023-06-23T01:04:31.599 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.583+0000 7f4b2a7fc700 1 -- 172.21.15.152:0/2255049616 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b30005bf0 con 0x7f4b3c06f170 2023-06-23T01:04:31.599 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.583+0000 7f4b2a7fc700 1 -- 172.21.15.152:0/2255049616 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b300070d0 con 0x7f4b3c06f170 2023-06-23T01:04:31.599 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.583+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c15b600 con 0x7f4b3c06f170 2023-06-23T01:04:31.599 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.583+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c15bbc0 con 0x7f4b3c06f170 2023-06-23T01:04:31.600 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.583+0000 7f4b2a7fc700 1 -- 172.21.15.152:0/2255049616 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4b3001c440 con 0x7f4b3c06f170 2023-06-23T01:04:31.600 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.583+0000 7f4b2a7fc700 1 -- 172.21.15.152:0/2255049616 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f4b300076c0 con 0x7f4b3c06f170 2023-06-23T01:04:31.600 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.584+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c04c1b0 con 0x7f4b3c06f170 2023-06-23T01:04:31.600 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.587+0000 7f4b2a7fc700 1 -- 172.21.15.152:0/2255049616 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f4b30017070 con 0x7f4b3c06f170 2023-06-23T01:04:31.677 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.674+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6ade832e-0e46-4c4f-8fb2-3a86a643e19b", "id": 7} v 0) v1 -- 0x7f4b3c0ce020 con 0x7f4b3c06f170 2023-06-23T01:04:31.681 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.678+0000 7f4b2a7fc700 1 -- 172.21.15.152:0/2255049616 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ade832e-0e46-4c4f-8fb2-3a86a643e19b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4b3001c880 con 0x7f4b3c06f170 2023-06-23T01:04:31.681 INFO:teuthology.orchestra.run.smithi152.stdout:7 2023-06-23T01:04:31.682 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 msgr2=0x7f4b3c0c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.682 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c0c9c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4b300011e0 tx=0x7f4b30007d30 comp rx=0 tx=0).stop 2023-06-23T01:04:31.682 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 shutdown_connections 2023-06-23T01:04:31.682 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b3c0cede0 0x7f4b3c0b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.683 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b3c06f170 0x7f4b3c0c9c30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.683 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 --2- 172.21.15.152:0/2255049616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b3c0b2550 0x7f4b3c145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.683 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 >> 172.21.15.152:0/2255049616 conn(0x7f4b3c0d02a0 msgr2=0x7f4b3c06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:31.683 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 shutdown_connections 2023-06-23T01:04:31.683 INFO:teuthology.orchestra.run.smithi152.stderr:2023-06-23T01:04:31.679+0000 7f4b40d4a700 1 -- 172.21.15.152:0/2255049616 wait complete. 2023-06-23T01:04:31.697 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-23T01:04:31.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-23T01:04:31.699 INFO:tasks.ceph.osd.0:Started 2023-06-23T01:04:31.700 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-23T01:04:31.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-23T01:04:31.702 INFO:tasks.ceph.osd.1:Started 2023-06-23T01:04:31.702 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-23T01:04:31.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-23T01:04:31.704 INFO:tasks.ceph.osd.2:Started 2023-06-23T01:04:31.705 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-23T01:04:31.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-23T01:04:31.708 INFO:tasks.ceph.osd.3:Started 2023-06-23T01:04:31.708 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-23T01:04:31.708 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-23T01:04:31.712 INFO:tasks.ceph.osd.4:Started 2023-06-23T01:04:31.712 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-23T01:04:31.713 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-23T01:04:31.715 INFO:tasks.ceph.osd.5:Started 2023-06-23T01:04:31.715 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-23T01:04:31.715 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-23T01:04:31.722 INFO:tasks.ceph.osd.6:Started 2023-06-23T01:04:31.723 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-23T01:04:31.723 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-23T01:04:31.730 INFO:tasks.ceph.osd.7:Started 2023-06-23T01:04:31.730 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:04:31.860 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:31.857+0000 7f9383017180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T01:04:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.911+0000 7fd00896e700 1 Processor -- start 2023-06-23T01:04:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.911+0000 7fd00896e700 1 -- start start 2023-06-23T01:04:31.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd00896e700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd0040cd620 0x7fd0040afd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd00896e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd004159560 0x7fd0040c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd00896e700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00406a320 con 0x7fd004159190 2023-06-23T01:04:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd00896e700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0040ca0c0 con 0x7fd0040cd620 2023-06-23T01:04:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd00896e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040ca3a0 con 0x7fd004159560 2023-06-23T01:04:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd001d9b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd0040cd620 0x7fd0040afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd002d9d700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd004159560 0x7fd0040c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd001d9b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd0040cd620 0x7fd0040afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.5:33526/0 (socket says 172.21.15.5:33526) 2023-06-23T01:04:31.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd001d9b700 1 -- 172.21.15.5:0/838228013 learned_addr learned my addr 172.21.15.5:0/838228013 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:31.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.912+0000 7fd00259c700 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd004159190 0x7fd0040ccf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.5:44436/0 (socket says 172.21.15.5:44436) 2023-06-23T01:04:31.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd002d9d700 1 -- 172.21.15.5:0/838228013 >> v2:172.21.15.152:3300/0 conn(0x7fd0040cd620 msgr2=0x7fd0040afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:31.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd002d9d700 1 --2- 172.21.15.5:0/838228013 >> v2:172.21.15.152:3300/0 conn(0x7fd0040cd620 0x7fd0040afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd002d9d700 1 -- 172.21.15.5:0/838228013 >> v1:172.21.15.152:6789/0 conn(0x7fd004159190 legacy=0x7fd0040ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd002d9d700 1 -- 172.21.15.5:0/838228013 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040ca680 con 0x7fd004159560 2023-06-23T01:04:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd001d9b700 1 --2- 172.21.15.5:0/838228013 >> v2:172.21.15.152:3300/0 conn(0x7fd0040cd620 0x7fd0040afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd00159a700 1 -- 172.21.15.5:0/838228013 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240339230 0 0) 0x7fd00406a320 con 0x7fd004159190 2023-06-23T01:04:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd002d9d700 1 --2- 172.21.15.5:0/838228013 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd004159560 0x7fd0040c9290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcff40098e0 tx=0x7fcff40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1e3e6cdab09379 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd00159a700 1 -- 172.21.15.5:0/838228013 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff400b640 con 0x7fd004159560 2023-06-23T01:04:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd00159a700 1 -- 172.21.15.5:0/838228013 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff400b7a0 con 0x7fd004159560 2023-06-23T01:04:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.913+0000 7fd00159a700 1 -- 172.21.15.5:0/838228013 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff400bb40 con 0x7fd004159560 2023-06-23T01:04:31.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.914+0000 7fd00896e700 1 -- 172.21.15.5:0/838228013 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd004159560 msgr2=0x7fd0040c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.914+0000 7fd00896e700 1 --2- 172.21.15.5:0/838228013 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd004159560 0x7fd0040c9290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcff40098e0 tx=0x7fcff40091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.914+0000 7fd00896e700 1 -- 172.21.15.5:0/838228013 shutdown_connections 2023-06-23T01:04:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.914+0000 7fd00896e700 1 --2- 172.21.15.5:0/838228013 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd004159560 0x7fd0040c9290 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.914+0000 7fd00896e700 1 --2- 172.21.15.5:0/838228013 >> v2:172.21.15.152:3300/0 conn(0x7fd0040cd620 0x7fd0040afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.914+0000 7fd00896e700 1 -- 172.21.15.5:0/838228013 >> 172.21.15.5:0/838228013 conn(0x7fd004068240 msgr2=0x7fd004068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.914+0000 7fd00896e700 1 -- 172.21.15.5:0/838228013 shutdown_connections 2023-06-23T01:04:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.914+0000 7fd00896e700 1 -- 172.21.15.5:0/838228013 wait complete. 2023-06-23T01:04:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.915+0000 7fd00896e700 1 Processor -- start 2023-06-23T01:04:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.915+0000 7fd00896e700 1 -- start start 2023-06-23T01:04:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00896e700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0040cd620 0x7fd00416a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00896e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd004159190 0x7fd0041505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00896e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd004159560 0x7fd0040c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00896e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040c33b0 con 0x7fd004159560 2023-06-23T01:04:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00896e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040c3690 con 0x7fd004159190 2023-06-23T01:04:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00896e700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd004169880 con 0x7fd0040cd620 2023-06-23T01:04:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd001d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd004159190 0x7fd0041505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd002d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd004159560 0x7fd0040c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd001d9b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd004159190 0x7fd0041505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.5:33534/0 (socket says 172.21.15.5:33534) 2023-06-23T01:04:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd001d9b700 1 -- 172.21.15.5:0/3272999076 learned_addr learned my addr 172.21.15.5:0/3272999076 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00259c700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0040cd620 0x7fd00416a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00259c700 1 -- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd004159190 msgr2=0x7fd0041505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00259c700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd004159190 0x7fd0041505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00259c700 1 -- 172.21.15.5:0/3272999076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd004159560 msgr2=0x7fd0040c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00259c700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd004159560 0x7fd0040c28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00259c700 1 -- 172.21.15.5:0/3272999076 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0094f0 con 0x7fd0040cd620 2023-06-23T01:04:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd001d9b700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd004159190 0x7fd0041505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.916+0000 7fd00259c700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0040cd620 0x7fd00416a080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00cd50 tx=0x7fcffc001b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.917+0000 7fcffa7fc700 1 -- 172.21.15.5:0/3272999076 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc005df0 con 0x7fd0040cd620 2023-06-23T01:04:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.917+0000 7fcffa7fc700 1 -- 172.21.15.5:0/3272999076 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc00a920 con 0x7fd0040cd620 2023-06-23T01:04:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.917+0000 7fcffa7fc700 1 -- 172.21.15.5:0/3272999076 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc00ace0 con 0x7fd0040cd620 2023-06-23T01:04:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.917+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff4009420 con 0x7fd0040cd620 2023-06-23T01:04:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.917+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd004169570 con 0x7fd0040cd620 2023-06-23T01:04:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.917+0000 7fcffa7fc700 1 -- 172.21.15.5:0/3272999076 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fcffc007300 con 0x7fd0040cd620 2023-06-23T01:04:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.917+0000 7fcffa7fc700 1 -- 172.21.15.5:0/3272999076 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fcffc00ae40 con 0x7fd0040cd620 2023-06-23T01:04:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.917+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0040c3800 con 0x7fd0040cd620 2023-06-23T01:04:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.921+0000 7fcffa7fc700 1 -- 172.21.15.5:0/3272999076 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7fcffc0075b0 con 0x7fd0040cd620 2023-06-23T01:04:31.952 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:31.948+0000 7fa30e5a8180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T01:04:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.991+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd004169da0 con 0x7fd0040cd620 2023-06-23T01:04:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.991+0000 7fcffa7fc700 1 -- 172.21.15.5:0/3272999076 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fcffc007750 con 0x7fd0040cd620 2023-06-23T01:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"fsid":"002f2efe-a784-4dcf-91c2-b9f520cd2c9c","created":"2023-06-23T01:04:28.422097+0000","modified":"2023-06-23T01:04:31.676217+0000","last_up_change":"0.000000","last_in_change":"2023-06-23T01:04:31.676217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2499edeb-6ba9-4392-b867-6a90f953982a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f2aa5a85-0aea-4bef-b45a-78297c1cd686","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"72aa25ff-c758-4780-9b74-fc1c316a3264","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"77dd6974-49b0-4288-b310-e7a0729ea2ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"80562fc5-676d-4cee-9978-4a94fca8df84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1db79ad4-997e-4516-a40b-90ee44d4957a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c18d6095-c8f0-4d76-94a8-f8de40a45195","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6ade832e-0e46-4c4f-8fb2-3a86a643e19b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:04:31.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0040cd620 msgr2=0x7fd00416a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:31.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0040cd620 0x7fd00416a080 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00cd50 tx=0x7fcffc001b00 comp rx=0 tx=0).stop 2023-06-23T01:04:31.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 shutdown_connections 2023-06-23T01:04:32.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd004159560 0x7fd0040c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:32.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd0040cd620 0x7fd00416a080 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:32.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 --2- 172.21.15.5:0/3272999076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd004159190 0x7fd0041505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:32.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 >> 172.21.15.5:0/3272999076 conn(0x7fd004068240 msgr2=0x7fd0040afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:32.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 shutdown_connections 2023-06-23T01:04:32.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:31.992+0000 7fd00896e700 1 -- 172.21.15.5:0/3272999076 wait complete. 2023-06-23T01:04:32.002 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:31.997+0000 7f9383017180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:04:32.008 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-23T01:04:32.009 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-23T01:04:32.092 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:32.089+0000 7f9383017180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T01:04:32.097 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:32.094+0000 7fa30e5a8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T01:04:32.193 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:32.190+0000 7fa30e5a8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T01:04:32.325 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-06-23T01:04:32.321+0000 7fc918a8e540 -1 Falling back to public interface 2023-06-23T01:04:32.333 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-06-23T01:04:32.330+0000 7f6e6e137540 -1 Falling back to public interface 2023-06-23T01:04:32.336 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-06-23T01:04:32.333+0000 7f627f9b6540 -1 Falling back to public interface 2023-06-23T01:04:32.345 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:32.342+0000 7f9383017180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T01:04:32.346 INFO:tasks.ceph.osd.1.smithi005.stderr:2023-06-23T01:04:32.342+0000 7f0dffdb7540 -1 Falling back to public interface 2023-06-23T01:04:32.346 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-06-23T01:04:32.343+0000 7f3620378540 -1 Falling back to public interface 2023-06-23T01:04:32.347 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-06-23T01:04:32.343+0000 7f361a7a1700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2023-06-23T01:04:32.347 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-06-23T01:04:32.344+0000 7f89c8d51540 -1 Falling back to public interface 2023-06-23T01:04:32.349 INFO:tasks.ceph.osd.0.smithi005.stderr:2023-06-23T01:04:32.347+0000 7fb8be938540 -1 Falling back to public interface 2023-06-23T01:04:32.351 INFO:tasks.ceph.osd.2.smithi005.stderr:2023-06-23T01:04:32.348+0000 7f90f4289540 -1 Falling back to public interface 2023-06-23T01:04:32.425 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:32.422+0000 7f9383017180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T01:04:32.454 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:32.450+0000 7fa30e5a8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T01:04:32.534 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:32.531+0000 7fa30e5a8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T01:04:32.814 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-23T01:04:32.924 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.921+0000 7f84151c1700 1 Processor -- start 2023-06-23T01:04:32.925 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.921+0000 7f84151c1700 1 -- start start 2023-06-23T01:04:32.927 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f84151c1700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f84100af5a0 0x7f84100ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:32.927 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f84151c1700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84100cb8d0 con 0x7f84100ca8c0 2023-06-23T01:04:32.927 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f84151c1700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8410159c80 con 0x7f841015cf30 2023-06-23T01:04:32.927 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f84151c1700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8410159930 con 0x7f84100af5a0 2023-06-23T01:04:32.928 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f840e59c700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f84100ca8c0 0x7f841015a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38012/0 (socket says 172.21.15.5:38012) 2023-06-23T01:04:32.928 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f840e59c700 1 -- 172.21.15.5:0/1135114031 learned_addr learned my addr 172.21.15.5:0/1135114031 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:32.928 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f840ed9d700 1 --2- 172.21.15.5:0/1135114031 >> v2:172.21.15.152:3300/0 conn(0x7f84100af5a0 0x7f84100ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:32.928 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f840dd9b700 1 -- 172.21.15.5:0/1135114031 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715669552 0 0) 0x7f84100cb8d0 con 0x7f84100ca8c0 2023-06-23T01:04:32.928 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f840dd9b700 1 -- 172.21.15.5:0/1135114031 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83fc003570 con 0x7f84100ca8c0 2023-06-23T01:04:32.928 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f840dd9b700 1 -- 172.21.15.5:0/1135114031 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332607353 0 0) 0x7f8410159c80 con 0x7f841015cf30 2023-06-23T01:04:32.928 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f840dd9b700 1 -- 172.21.15.5:0/1135114031 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84100cb8d0 con 0x7f841015cf30 2023-06-23T01:04:32.929 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.922+0000 7f840ed9d700 1 -- 172.21.15.5:0/1135114031 >> v1:172.21.15.152:6789/0 conn(0x7f841015cf30 legacy=0x7f8410068d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:32.929 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f840ed9d700 1 -- 172.21.15.5:0/1135114031 >> v1:172.21.15.5:6789/0 conn(0x7f84100ca8c0 legacy=0x7f841015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:32.929 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f840ed9d700 1 -- 172.21.15.5:0/1135114031 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84100cb030 con 0x7f84100af5a0 2023-06-23T01:04:32.929 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f840ed9d700 1 --2- 172.21.15.5:0/1135114031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100af5a0 0x7f84100ceb00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f84000098d0 tx=0x7f8400009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1130347522c75c1a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:32.929 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f840dd9b700 1 -- 172.21.15.5:0/1135114031 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400005c10 con 0x7f84100af5a0 2023-06-23T01:04:32.929 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f840dd9b700 1 -- 172.21.15.5:0/1135114031 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8400005d70 con 0x7f84100af5a0 2023-06-23T01:04:32.929 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f840dd9b700 1 -- 172.21.15.5:0/1135114031 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400006260 con 0x7f84100af5a0 2023-06-23T01:04:32.929 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f84151c1700 1 -- 172.21.15.5:0/1135114031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100af5a0 msgr2=0x7f84100ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:32.930 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f84151c1700 1 --2- 172.21.15.5:0/1135114031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100af5a0 0x7f84100ceb00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f84000098d0 tx=0x7f8400009120 comp rx=0 tx=0).stop 2023-06-23T01:04:32.930 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f84151c1700 1 -- 172.21.15.5:0/1135114031 shutdown_connections 2023-06-23T01:04:32.930 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f84151c1700 1 --2- 172.21.15.5:0/1135114031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100af5a0 0x7f84100ceb00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:32.930 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f84151c1700 1 -- 172.21.15.5:0/1135114031 >> 172.21.15.5:0/1135114031 conn(0x7f84100d0280 msgr2=0x7f84100d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:32.930 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f84151c1700 1 -- 172.21.15.5:0/1135114031 shutdown_connections 2023-06-23T01:04:32.930 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.923+0000 7f84151c1700 1 -- 172.21.15.5:0/1135114031 wait complete. 2023-06-23T01:04:32.932 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.925+0000 7f84151c1700 1 Processor -- start 2023-06-23T01:04:32.932 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.925+0000 7f84151c1700 1 -- start start 2023-06-23T01:04:32.933 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.925+0000 7f84151c1700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84100af5a0 0x7f8410149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:32.933 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.925+0000 7f84151c1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100ca8c0 0x7f84100b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:32.933 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.925+0000 7f84151c1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015cf30 0x7f84100c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:32.933 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.925+0000 7f84151c1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100b3020 con 0x7f841015cf30 2023-06-23T01:04:32.933 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.925+0000 7f84151c1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100b35a0 con 0x7f84100ca8c0 2023-06-23T01:04:32.933 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.925+0000 7f84151c1700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c9cc0 con 0x7f84100af5a0 2023-06-23T01:04:32.933 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015cf30 0x7f84100c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:32.934 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015cf30 0x7f84100c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49580/0 (socket says 172.21.15.5:49580) 2023-06-23T01:04:32.934 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 -- 172.21.15.5:0/3410689369 learned_addr learned my addr 172.21.15.5:0/3410689369 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:32.934 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840e59c700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100ca8c0 0x7f84100b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:32.934 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840ed9d700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84100af5a0 0x7f8410149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:32.934 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 -- 172.21.15.5:0/3410689369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84100af5a0 msgr2=0x7f8410149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:32.934 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84100af5a0 0x7f8410149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:32.934 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 -- 172.21.15.5:0/3410689369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100ca8c0 msgr2=0x7f84100b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:32.934 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100ca8c0 0x7f84100b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:32.935 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 -- 172.21.15.5:0/3410689369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000093c0 con 0x7f841015cf30 2023-06-23T01:04:32.935 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f840f59e700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015cf30 0x7f84100c9350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f840800c860 tx=0x7f8408002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:32.935 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f8406ffd700 1 -- 172.21.15.5:0/3410689369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408002030 con 0x7f841015cf30 2023-06-23T01:04:32.935 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f8406ffd700 1 -- 172.21.15.5:0/3410689369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84080021b0 con 0x7f841015cf30 2023-06-23T01:04:32.935 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84100c9fa0 con 0x7f841015cf30 2023-06-23T01:04:32.935 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f8406ffd700 1 -- 172.21.15.5:0/3410689369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408012630 con 0x7f841015cf30 2023-06-23T01:04:32.935 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841014a210 con 0x7f841015cf30 2023-06-23T01:04:32.935 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f8406ffd700 1 -- 172.21.15.5:0/3410689369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8408012a20 con 0x7f841015cf30 2023-06-23T01:04:32.936 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.926+0000 7f8406ffd700 1 -- 172.21.15.5:0/3410689369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f84080014f0 con 0x7f841015cf30 2023-06-23T01:04:32.936 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.927+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841004c1b0 con 0x7f841015cf30 2023-06-23T01:04:32.936 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:32.931+0000 7f8406ffd700 1 -- 172.21.15.5:0/3410689369 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71365 (secure 0 0 0) 0x7f8408012cd0 con 0x7f841015cf30 2023-06-23T01:04:33.004 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.001+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84100b36e0 con 0x7f841015cf30 2023-06-23T01:04:33.005 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.002+0000 7f8406ffd700 1 -- 172.21.15.5:0/3410689369 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f84080019a0 con 0x7f841015cf30 2023-06-23T01:04:33.005 INFO:teuthology.misc.health.smithi005.stdout: 2023-06-23T01:04:33.005 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":10,"fsid":"002f2efe-a784-4dcf-91c2-b9f520cd2c9c","created":"2023-06-23T01:04:28.422097+0000","modified":"2023-06-23T01:04:31.676217+0000","last_up_change":"0.000000","last_in_change":"2023-06-23T01:04:31.676217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2499edeb-6ba9-4392-b867-6a90f953982a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f2aa5a85-0aea-4bef-b45a-78297c1cd686","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"72aa25ff-c758-4780-9b74-fc1c316a3264","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"77dd6974-49b0-4288-b310-e7a0729ea2ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"80562fc5-676d-4cee-9978-4a94fca8df84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1db79ad4-997e-4516-a40b-90ee44d4957a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c18d6095-c8f0-4d76-94a8-f8de40a45195","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6ade832e-0e46-4c4f-8fb2-3a86a643e19b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:04:33.006 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.002+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015cf30 msgr2=0x7f84100c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:33.006 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.002+0000 7f84151c1700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015cf30 0x7f84100c9350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f840800c860 tx=0x7f8408002700 comp rx=0 tx=0).stop 2023-06-23T01:04:33.007 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.003+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 shutdown_connections 2023-06-23T01:04:33.007 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.003+0000 7f84151c1700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015cf30 0x7f84100c9350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:33.007 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.003+0000 7f84151c1700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f84100af5a0 0x7f8410149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:33.008 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.003+0000 7f84151c1700 1 --2- 172.21.15.5:0/3410689369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84100ca8c0 0x7f84100b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:33.008 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.003+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 >> 172.21.15.5:0/3410689369 conn(0x7f84100d0280 msgr2=0x7f8410158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:33.009 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.003+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 shutdown_connections 2023-06-23T01:04:33.009 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:33.003+0000 7f84151c1700 1 -- 172.21.15.5:0/3410689369 wait complete. 2023-06-23T01:04:33.017 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-23T01:04:33.257 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:33.254+0000 7f9383017180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T01:04:33.330 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:33.327+0000 7f9383017180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T01:04:33.405 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:33.401+0000 7f9383017180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T01:04:33.410 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:33.407+0000 7fa30e5a8180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T01:04:33.487 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:33.484+0000 7fa30e5a8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T01:04:33.539 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:33.536+0000 7f9383017180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T01:04:33.565 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:33.562+0000 7fa30e5a8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T01:04:33.624 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:33.621+0000 7f9383017180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T01:04:33.705 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:33.702+0000 7fa30e5a8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T01:04:33.775 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:33.772+0000 7f9383017180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T01:04:33.794 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:33.790+0000 7fa30e5a8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T01:04:33.852 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:33.849+0000 7f9383017180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T01:04:33.927 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:33.924+0000 7f9383017180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T01:04:33.953 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:33.950+0000 7fa30e5a8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T01:04:34.003 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:34.000+0000 7f9383017180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T01:04:34.034 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:34.030+0000 7fa30e5a8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T01:04:34.114 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:34.111+0000 7fa30e5a8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T01:04:34.193 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-06-23T01:04:34.190+0000 7fa30e5a8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T01:04:35.028 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:35.025+0000 7f933c418700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-23T01:04:35.029 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-06-23T01:04:35.025+0000 7f933c418700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-23T01:04:35.823 INFO:tasks.ceph.osd.1.smithi005.stderr:2023-06-23T01:04:35.820+0000 7f0dffdb7540 -1 osd.1 0 log_to_monitors true 2023-06-23T01:04:35.831 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-06-23T01:04:35.828+0000 7f89c8d51540 -1 osd.7 0 log_to_monitors true 2023-06-23T01:04:35.845 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-06-23T01:04:35.842+0000 7f3620378540 -1 osd.3 0 log_to_monitors true 2023-06-23T01:04:35.848 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-06-23T01:04:35.845+0000 7f6e6e137540 -1 osd.6 0 log_to_monitors true 2023-06-23T01:04:35.852 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-06-23T01:04:35.849+0000 7f627f9b6540 -1 osd.4 0 log_to_monitors true 2023-06-23T01:04:35.860 INFO:tasks.ceph.osd.0.smithi005.stderr:2023-06-23T01:04:35.857+0000 7fb8be938540 -1 osd.0 0 log_to_monitors true 2023-06-23T01:04:35.863 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-06-23T01:04:35.860+0000 7fc918a8e540 -1 osd.5 0 log_to_monitors true 2023-06-23T01:04:35.870 INFO:tasks.ceph.osd.2.smithi005.stderr:2023-06-23T01:04:35.867+0000 7f90f4289540 -1 osd.2 0 log_to_monitors true 2023-06-23T01:04:38.422 INFO:tasks.ceph.osd.1.smithi005.stderr:2023-06-23T01:04:38.419+0000 7f0df69d9700 -1 osd.1 0 waiting for initial osdmap 2023-06-23T01:04:38.424 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-06-23T01:04:38.421+0000 7f3616f9a700 -1 osd.3 0 waiting for initial osdmap 2023-06-23T01:04:38.427 INFO:tasks.ceph.osd.1.smithi005.stderr:2023-06-23T01:04:38.424+0000 7f0def7d0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:04:38.428 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-06-23T01:04:38.424+0000 7f360fd91700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:04:38.428 INFO:tasks.ceph.osd.0.smithi005.stderr:2023-06-23T01:04:38.425+0000 7fb8b555a700 -1 osd.0 0 waiting for initial osdmap 2023-06-23T01:04:38.430 INFO:tasks.ceph.osd.2.smithi005.stderr:2023-06-23T01:04:38.427+0000 7f90eaeab700 -1 osd.2 0 waiting for initial osdmap 2023-06-23T01:04:38.432 INFO:tasks.ceph.osd.0.smithi005.stderr:2023-06-23T01:04:38.429+0000 7fb8ae351700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:04:38.434 INFO:tasks.ceph.osd.2.smithi005.stderr:2023-06-23T01:04:38.431+0000 7f90e3ca2700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:04:39.106 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-06-23T01:04:39.103+0000 7f62765d8700 -1 osd.4 0 waiting for initial osdmap 2023-06-23T01:04:39.109 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-06-23T01:04:39.106+0000 7fc90f6b0700 -1 osd.5 0 waiting for initial osdmap 2023-06-23T01:04:39.112 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-06-23T01:04:39.109+0000 7f626f3cf700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:04:39.118 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-06-23T01:04:39.115+0000 7fc9084a7700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:04:39.126 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-06-23T01:04:39.124+0000 7f89bf973700 -1 osd.7 0 waiting for initial osdmap 2023-06-23T01:04:39.127 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-06-23T01:04:39.124+0000 7f6e64d59700 -1 osd.6 0 waiting for initial osdmap 2023-06-23T01:04:39.133 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-06-23T01:04:39.130+0000 7f89b876a700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:04:39.133 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-06-23T01:04:39.130+0000 7f6e5db50700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T01:04:39.824 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-23T01:04:39.948 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.944+0000 7f7dfb0ea700 1 Processor -- start 2023-06-23T01:04:39.948 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.945+0000 7f7dfb0ea700 1 -- start start 2023-06-23T01:04:39.950 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.945+0000 7f7dfb0ea700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7df406bbb0 0x7f7df40b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:39.950 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.945+0000 7f7dfb0ea700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df40b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:39.950 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.945+0000 7f7dfb0ea700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df415bb40 con 0x7f7df4154ad0 2023-06-23T01:04:39.950 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.945+0000 7f7dfb0ea700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df415af30 con 0x7f7df406bbb0 2023-06-23T01:04:39.951 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.945+0000 7f7dfb0ea700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df415b210 con 0x7f7df4154ea0 2023-06-23T01:04:39.951 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df9687700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df40b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:39.951 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df3fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7df406bbb0 0x7f7df40b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:39.951 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df3fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7df406bbb0 0x7f7df40b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.5:33582/0 (socket says 172.21.15.5:33582) 2023-06-23T01:04:39.951 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df9687700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df40b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.5:50796/0 (socket says 172.21.15.5:50796) 2023-06-23T01:04:39.951 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df3fff700 1 -- 172.21.15.5:0/1860242137 learned_addr learned my addr 172.21.15.5:0/1860242137 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:39.951 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df8e86700 1 --1- 172.21.15.5:0/1860242137 >> v1:172.21.15.152:6789/0 conn(0x7f7df4154ad0 0x7f7df406b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.5:57160/0 (socket says 172.21.15.5:57160) 2023-06-23T01:04:39.952 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df3fff700 1 -- 172.21.15.5:0/1860242137 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 msgr2=0x7f7df40b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:39.952 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df3fff700 1 --2- 172.21.15.5:0/1860242137 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df40b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:39.952 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df3fff700 1 -- 172.21.15.5:0/1860242137 >> v1:172.21.15.152:6789/0 conn(0x7f7df4154ad0 legacy=0x7f7df406b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:39.952 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df3fff700 1 -- 172.21.15.5:0/1860242137 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df415b4f0 con 0x7f7df406bbb0 2023-06-23T01:04:39.952 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df37fe700 1 -- 172.21.15.5:0/1860242137 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212158417 0 0) 0x7f7df415bb40 con 0x7f7df4154ad0 2023-06-23T01:04:39.952 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df3fff700 1 --2- 172.21.15.5:0/1860242137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 0x7f7df40b3060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009f20 tx=0x7f7de4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=639c97dcecfb69a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:39.952 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df37fe700 1 -- 172.21.15.5:0/1860242137 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de400e070 con 0x7f7df406bbb0 2023-06-23T01:04:39.953 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df37fe700 1 -- 172.21.15.5:0/1860242137 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de400b9d0 con 0x7f7df406bbb0 2023-06-23T01:04:39.953 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.946+0000 7f7df37fe700 1 -- 172.21.15.5:0/1860242137 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de40092e0 con 0x7f7df406bbb0 2023-06-23T01:04:39.953 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.947+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/1860242137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 msgr2=0x7f7df40b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:39.953 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.947+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/1860242137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 0x7f7df40b3060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009f20 tx=0x7f7de4009840 comp rx=0 tx=0).stop 2023-06-23T01:04:39.953 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.947+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/1860242137 shutdown_connections 2023-06-23T01:04:39.953 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.947+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/1860242137 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df40b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:39.953 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.947+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/1860242137 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 0x7f7df40b3060 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:39.953 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.947+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/1860242137 >> 172.21.15.5:0/1860242137 conn(0x7f7df4067660 msgr2=0x7f7df40afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:39.954 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.947+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/1860242137 shutdown_connections 2023-06-23T01:04:39.954 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.947+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/1860242137 wait complete. 2023-06-23T01:04:39.956 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.948+0000 7f7dfb0ea700 1 Processor -- start 2023-06-23T01:04:39.956 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.948+0000 7f7dfb0ea700 1 -- start start 2023-06-23T01:04:39.956 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7dfb0ea700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 0x7f7df4150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:39.956 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7dfb0ea700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7df4154ad0 0x7f7df40686d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:39.956 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7dfb0ea700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df416cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:39.956 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7dfb0ea700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40c8910 con 0x7f7df4154ad0 2023-06-23T01:04:39.957 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7dfb0ea700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df416d4e0 con 0x7f7df406bbb0 2023-06-23T01:04:39.957 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7dfb0ea700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df416d6f0 con 0x7f7df4154ea0 2023-06-23T01:04:39.957 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7df4154ad0 0x7f7df40686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:39.957 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7df4154ad0 0x7f7df40686d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43792/0 (socket says 172.21.15.5:43792) 2023-06-23T01:04:39.957 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df3fff700 1 -- 172.21.15.5:0/3840685808 learned_addr learned my addr 172.21.15.5:0/3840685808 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:39.957 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df8e86700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 0x7f7df4150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:39.957 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df9687700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df416cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:39.958 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df3fff700 1 -- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 msgr2=0x7f7df416cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:39.958 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df3fff700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df416cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:39.958 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df3fff700 1 -- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 msgr2=0x7f7df4150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:39.958 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df3fff700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 0x7f7df4150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:39.958 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.949+0000 7f7df3fff700 1 -- 172.21.15.5:0/3840685808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4009ae0 con 0x7f7df4154ad0 2023-06-23T01:04:39.958 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.950+0000 7f7df3fff700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7df4154ad0 0x7f7df40686d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7df406ec50 tx=0x7f7de4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:39.958 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.950+0000 7f7df0ff9700 1 -- 172.21.15.5:0/3840685808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4008870 con 0x7f7df4154ad0 2023-06-23T01:04:39.959 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.950+0000 7f7df0ff9700 1 -- 172.21.15.5:0/3840685808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de40089d0 con 0x7f7df4154ad0 2023-06-23T01:04:39.959 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.950+0000 7f7df0ff9700 1 -- 172.21.15.5:0/3840685808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4008d90 con 0x7f7df4154ad0 2023-06-23T01:04:39.959 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.950+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df416d9d0 con 0x7f7df4154ad0 2023-06-23T01:04:39.959 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.950+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df40c8b20 con 0x7f7df4154ad0 2023-06-23T01:04:39.959 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.951+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df41513a0 con 0x7f7df4154ad0 2023-06-23T01:04:39.960 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.951+0000 7f7df0ff9700 1 -- 172.21.15.5:0/3840685808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7f7de4003040 con 0x7f7df4154ad0 2023-06-23T01:04:39.960 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.952+0000 7f7df0ff9700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f7dd4066e00 0x7f7dd40692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:39.960 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.952+0000 7f7df0ff9700 1 -- 172.21.15.5:0/3840685808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f7de40857e0 con 0x7f7df4154ad0 2023-06-23T01:04:39.960 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.952+0000 7f7df8e86700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f7dd4066e00 0x7f7dd40692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:39.961 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.952+0000 7f7df8e86700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f7dd4066e00 0x7f7dd40692b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007fd0 tx=0x7f7ddc007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:39.961 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:39.956+0000 7f7df0ff9700 1 -- 172.21.15.5:0/3840685808 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f7de4053170 con 0x7f7df4154ad0 2023-06-23T01:04:40.159 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.156+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7df404c1b0 con 0x7f7df4154ad0 2023-06-23T01:04:40.160 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.157+0000 7f7df0ff9700 1 -- 172.21.15.5:0/3840685808 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10348 (secure 0 0 0) 0x7f7de4056790 con 0x7f7df4154ad0 2023-06-23T01:04:40.160 INFO:teuthology.misc.health.smithi005.stdout: 2023-06-23T01:04:40.160 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":13,"fsid":"002f2efe-a784-4dcf-91c2-b9f520cd2c9c","created":"2023-06-23T01:04:28.422097+0000","modified":"2023-06-23T01:04:39.254387+0000","last_up_change":"2023-06-23T01:04:39.254387+0000","last_in_change":"2023-06-23T01:04:31.676217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"2499edeb-6ba9-4392-b867-6a90f953982a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":87985},{"type":"v1","addr":"172.21.15.5:6817","nonce":87985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":87985},{"type":"v1","addr":"172.21.15.5:6819","nonce":87985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":87985},{"type":"v1","addr":"172.21.15.5:6826","nonce":87985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":87985},{"type":"v1","addr":"172.21.15.5:6822","nonce":87985}]},"public_addr":"172.21.15.5:6817/87985","cluster_addr":"172.21.15.5:6819/87985","heartbeat_back_addr":"172.21.15.5:6826/87985","heartbeat_front_addr":"172.21.15.5:6822/87985","state":["exists","up"]},{"osd":1,"uuid":"f2aa5a85-0aea-4bef-b45a-78297c1cd686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":87988},{"type":"v1","addr":"172.21.15.5:6801","nonce":87988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":87988},{"type":"v1","addr":"172.21.15.5:6804","nonce":87988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":87988},{"type":"v1","addr":"172.21.15.5:6812","nonce":87988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":87988},{"type":"v1","addr":"172.21.15.5:6808","nonce":87988}]},"public_addr":"172.21.15.5:6801/87988","cluster_addr":"172.21.15.5:6804/87988","heartbeat_back_addr":"172.21.15.5:6812/87988","heartbeat_front_addr":"172.21.15.5:6808/87988","state":["exists","up"]},{"osd":2,"uuid":"72aa25ff-c758-4780-9b74-fc1c316a3264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6821","nonce":87990},{"type":"v1","addr":"172.21.15.5:6823","nonce":87990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6825","nonce":87990},{"type":"v1","addr":"172.21.15.5:6827","nonce":87990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":87990},{"type":"v1","addr":"172.21.15.5:6831","nonce":87990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":87990},{"type":"v1","addr":"172.21.15.5:6829","nonce":87990}]},"public_addr":"172.21.15.5:6823/87990","cluster_addr":"172.21.15.5:6827/87990","heartbeat_back_addr":"172.21.15.5:6831/87990","heartbeat_front_addr":"172.21.15.5:6829/87990","state":["exists","up"]},{"osd":3,"uuid":"77dd6974-49b0-4288-b310-e7a0729ea2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":87991},{"type":"v1","addr":"172.21.15.5:6805","nonce":87991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":87991},{"type":"v1","addr":"172.21.15.5:6809","nonce":87991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":87991},{"type":"v1","addr":"172.21.15.5:6815","nonce":87991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":87991},{"type":"v1","addr":"172.21.15.5:6813","nonce":87991}]},"public_addr":"172.21.15.5:6805/87991","cluster_addr":"172.21.15.5:6809/87991","heartbeat_back_addr":"172.21.15.5:6815/87991","heartbeat_front_addr":"172.21.15.5:6813/87991","state":["exists","up"]},{"osd":4,"uuid":"80562fc5-676d-4cee-9978-4a94fca8df84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":88231},{"type":"v1","addr":"172.21.15.152:6817","nonce":88231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":88231},{"type":"v1","addr":"172.21.15.152:6819","nonce":88231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":88231},{"type":"v1","addr":"172.21.15.152:6823","nonce":88231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":88231},{"type":"v1","addr":"172.21.15.152:6821","nonce":88231}]},"public_addr":"172.21.15.152:6817/88231","cluster_addr":"172.21.15.152:6819/88231","heartbeat_back_addr":"172.21.15.152:6823/88231","heartbeat_front_addr":"172.21.15.152:6821/88231","state":["exists","up"]},{"osd":5,"uuid":"1db79ad4-997e-4516-a40b-90ee44d4957a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":88215},{"type":"v1","addr":"172.21.15.152:6801","nonce":88215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":88215},{"type":"v1","addr":"172.21.15.152:6803","nonce":88215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":88215},{"type":"v1","addr":"172.21.15.152:6807","nonce":88215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":88215},{"type":"v1","addr":"172.21.15.152:6805","nonce":88215}]},"public_addr":"172.21.15.152:6801/88215","cluster_addr":"172.21.15.152:6803/88215","heartbeat_back_addr":"172.21.15.152:6807/88215","heartbeat_front_addr":"172.21.15.152:6805/88215","state":["exists","up"]},{"osd":6,"uuid":"c18d6095-c8f0-4d76-94a8-f8de40a45195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":88216},{"type":"v1","addr":"172.21.15.152:6809","nonce":88216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":88216},{"type":"v1","addr":"172.21.15.152:6811","nonce":88216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":88216},{"type":"v1","addr":"172.21.15.152:6815","nonce":88216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":88216},{"type":"v1","addr":"172.21.15.152:6813","nonce":88216}]},"public_addr":"172.21.15.152:6809/88216","cluster_addr":"172.21.15.152:6811/88216","heartbeat_back_addr":"172.21.15.152:6815/88216","heartbeat_front_addr":"172.21.15.152:6813/88216","state":["exists","up"]},{"osd":7,"uuid":"6ade832e-0e46-4c4f-8fb2-3a86a643e19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":88246},{"type":"v1","addr":"172.21.15.152:6825","nonce":88246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":88246},{"type":"v1","addr":"172.21.15.152:6827","nonce":88246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":88246},{"type":"v1","addr":"172.21.15.152:6831","nonce":88246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":88246},{"type":"v1","addr":"172.21.15.152:6829","nonce":88246}]},"public_addr":"172.21.15.152:6825/88246","cluster_addr":"172.21.15.152:6827/88246","heartbeat_back_addr":"172.21.15.152:6831/88246","heartbeat_front_addr":"172.21.15.152:6829/88246","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:04:40.162 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f7dd4066e00 msgr2=0x7f7dd40692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:40.162 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f7dd4066e00 0x7f7dd40692b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc007fd0 tx=0x7f7ddc007d50 comp rx=0 tx=0).stop 2023-06-23T01:04:40.162 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7df4154ad0 msgr2=0x7f7df40686d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:40.162 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7df4154ad0 0x7f7df40686d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7df406ec50 tx=0x7f7de4018000 comp rx=0 tx=0).stop 2023-06-23T01:04:40.162 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 shutdown_connections 2023-06-23T01:04:40.163 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f7dd4066e00 0x7f7dd40692b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.163 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7df4154ea0 0x7f7df416cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.163 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7df406bbb0 0x7f7df4150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.163 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 --2- 172.21.15.5:0/3840685808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7df4154ad0 0x7f7df40686d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.163 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 >> 172.21.15.5:0/3840685808 conn(0x7f7df4067660 msgr2=0x7f7df40cb940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:40.163 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 shutdown_connections 2023-06-23T01:04:40.163 INFO:teuthology.misc.health.smithi005.stderr:2023-06-23T01:04:40.159+0000 7f7dfb0ea700 1 -- 172.21.15.5:0/3840685808 wait complete. 2023-06-23T01:04:40.173 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-23T01:04:40.173 INFO:tasks.ceph:Creating RBD pool 2023-06-23T01:04:40.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-23T01:04:40.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.288+0000 7f221e555700 1 Processor -- start 2023-06-23T01:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.289+0000 7f221e555700 1 -- start start 2023-06-23T01:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.289+0000 7f221e555700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2218069e50 0x7f221815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.289+0000 7f221e555700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f22180c88c0 0x7f221806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.289+0000 7f221e555700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f221806a220 0x7f221806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.289+0000 7f221e555700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22180ac2a0 con 0x7f22180c88c0 2023-06-23T01:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.289+0000 7f221e555700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f221815aab0 con 0x7f221806a220 2023-06-23T01:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.289+0000 7f221e555700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221815ad90 con 0x7f2218069e50 2023-06-23T01:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.289+0000 7f22177fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f22180c88c0 0x7f221806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f22177fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f22180c88c0 0x7f221806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43808/0 (socket says 172.21.15.5:43808) 2023-06-23T01:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f22177fe700 1 -- 172.21.15.5:0/1994971638 learned_addr learned my addr 172.21.15.5:0/1994971638 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f221caf2700 1 --2- 172.21.15.5:0/1994971638 >> v2:172.21.15.152:3300/0 conn(0x7f221806a220 0x7f221806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f2217fff700 1 --2- 172.21.15.5:0/1994971638 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2218069e50 0x7f221815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f22177fe700 1 -- 172.21.15.5:0/1994971638 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2218069e50 msgr2=0x7f221815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f22177fe700 1 --2- 172.21.15.5:0/1994971638 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2218069e50 0x7f221815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f22177fe700 1 -- 172.21.15.5:0/1994971638 >> v2:172.21.15.152:3300/0 conn(0x7f221806a220 msgr2=0x7f221806c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f22177fe700 1 --2- 172.21.15.5:0/1994971638 >> v2:172.21.15.152:3300/0 conn(0x7f221806a220 0x7f221806c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f22177fe700 1 -- 172.21.15.5:0/1994971638 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221815b070 con 0x7f22180c88c0 2023-06-23T01:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f22177fe700 1 --2- 172.21.15.5:0/1994971638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22180c88c0 0x7f221806c000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f221806dad0 tx=0x7f220c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75a2ec6c60475da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f2216ffd700 1 -- 172.21.15.5:0/1994971638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c00b860 con 0x7f22180c88c0 2023-06-23T01:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f2216ffd700 1 -- 172.21.15.5:0/1994971638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c00b9c0 con 0x7f22180c88c0 2023-06-23T01:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.290+0000 7f2216ffd700 1 -- 172.21.15.5:0/1994971638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c00bd60 con 0x7f22180c88c0 2023-06-23T01:04:40.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 -- 172.21.15.5:0/1994971638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22180c88c0 msgr2=0x7f221806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:40.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 --2- 172.21.15.5:0/1994971638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22180c88c0 0x7f221806c000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f221806dad0 tx=0x7f220c009250 comp rx=0 tx=0).stop 2023-06-23T01:04:40.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 -- 172.21.15.5:0/1994971638 shutdown_connections 2023-06-23T01:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 --2- 172.21.15.5:0/1994971638 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2218069e50 0x7f221815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 --2- 172.21.15.5:0/1994971638 >> v2:172.21.15.152:3300/0 conn(0x7f221806a220 0x7f221806c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 --2- 172.21.15.5:0/1994971638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22180c88c0 0x7f221806c000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 -- 172.21.15.5:0/1994971638 >> 172.21.15.5:0/1994971638 conn(0x7f221806e830 msgr2=0x7f221806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 -- 172.21.15.5:0/1994971638 shutdown_connections 2023-06-23T01:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.291+0000 7f221e555700 1 -- 172.21.15.5:0/1994971638 wait complete. 2023-06-23T01:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.292+0000 7f221e555700 1 Processor -- start 2023-06-23T01:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.292+0000 7f221e555700 1 -- start start 2023-06-23T01:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f221e555700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2218069e50 0x7f221813d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f221e555700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221806a220 0x7f2218158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f221e555700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22180c88c0 0x7f2218158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f221e555700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180ccb00 con 0x7f221806a220 2023-06-23T01:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f221e555700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180cd4f0 con 0x7f2218069e50 2023-06-23T01:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f221e555700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22180cbef0 con 0x7f22180c88c0 2023-06-23T01:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221806a220 0x7f2218158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221806a220 0x7f2218158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43812/0 (socket says 172.21.15.5:43812) 2023-06-23T01:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 -- 172.21.15.5:0/2172621970 learned_addr learned my addr 172.21.15.5:0/2172621970 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f2217fff700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2218069e50 0x7f221813d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f221caf2700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22180c88c0 0x7f2218158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 -- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22180c88c0 msgr2=0x7f2218158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22180c88c0 0x7f2218158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 -- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2218069e50 msgr2=0x7f221813d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2218069e50 0x7f221813d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 -- 172.21.15.5:0/2172621970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c0094f0 con 0x7f221806a220 2023-06-23T01:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f22177fe700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221806a220 0x7f2218158130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f221806bdf0 tx=0x7f220c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f21fffff700 1 -- 172.21.15.5:0/2172621970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c011e40 con 0x7f221806a220 2023-06-23T01:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.293+0000 7f21fffff700 1 -- 172.21.15.5:0/2172621970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c01a450 con 0x7f221806a220 2023-06-23T01:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.294+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22180cc1d0 con 0x7f221806a220 2023-06-23T01:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.294+0000 7f21fffff700 1 -- 172.21.15.5:0/2172621970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c01a810 con 0x7f221806a220 2023-06-23T01:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.294+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22180cc790 con 0x7f221806a220 2023-06-23T01:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.295+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221804c1b0 con 0x7f221806a220 2023-06-23T01:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.295+0000 7f21fffff700 1 -- 172.21.15.5:0/2172621970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7f220c01abc0 con 0x7f221806a220 2023-06-23T01:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.295+0000 7f21fffff700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f2204066e00 0x7f22040692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.295+0000 7f21fffff700 1 -- 172.21.15.5:0/2172621970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f220c085a00 con 0x7f221806a220 2023-06-23T01:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.295+0000 7f2217fff700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f2204066e00 0x7f22040692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.296+0000 7f2217fff700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f2204066e00 0x7f22040692b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2208006fd0 tx=0x7f2208008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:40.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.300+0000 7f21fffff700 1 -- 172.21.15.5:0/2172621970 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f220c053240 con 0x7f221806a220 2023-06-23T01:04:40.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:40.495+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f2218070650 con 0x7f221806a220 2023-06-23T01:04:41.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.488+0000 7f21fffff700 1 -- 172.21.15.5:0/2172621970 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f220c056860 con 0x7f221806a220 2023-06-23T01:04:41.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f2204066e00 msgr2=0x7f22040692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:41.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f2204066e00 0x7f22040692b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2208006fd0 tx=0x7f2208008040 comp rx=0 tx=0).stop 2023-06-23T01:04:41.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221806a220 msgr2=0x7f2218158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:41.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221806a220 0x7f2218158130 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f221806bdf0 tx=0x7f220c017000 comp rx=0 tx=0).stop 2023-06-23T01:04:41.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 shutdown_connections 2023-06-23T01:04:41.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f2204066e00 0x7f22040692b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:41.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22180c88c0 0x7f2218158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:41.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2218069e50 0x7f221813d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:41.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 --2- 172.21.15.5:0/2172621970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221806a220 0x7f2218158130 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:41.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.491+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 >> 172.21.15.5:0/2172621970 conn(0x7f221806e830 msgr2=0x7f221806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:41.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.492+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 shutdown_connections 2023-06-23T01:04:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.492+0000 7f221e555700 1 -- 172.21.15.5:0/2172621970 wait complete. 2023-06-23T01:04:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'rbd' created 2023-06-23T01:04:41.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-23T01:04:41.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.641+0000 7fc9e17db700 1 Processor -- start 2023-06-23T01:04:41.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.641+0000 7fc9e17db700 1 -- start start 2023-06-23T01:04:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9e17db700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc9dc0cd380 0x7fc9dc15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9e17db700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9dc0afce0 con 0x7fc9dc153b50 2023-06-23T01:04:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9e17db700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9dc15dbb0 con 0x7fc9dc153780 2023-06-23T01:04:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9e17db700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9dc15cfa0 con 0x7fc9dc0cd380 2023-06-23T01:04:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9db7fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc9dc0cd380 0x7fc9dc15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9da7fc700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc9dc153b50 0x7fc9dc0ccc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40156/0 (socket says 172.21.15.5:40156) 2023-06-23T01:04:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9da7fc700 1 -- 172.21.15.5:0/1656672022 learned_addr learned my addr 172.21.15.5:0/1656672022 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9db7fe700 1 -- 172.21.15.5:0/1656672022 >> v1:172.21.15.152:6789/0 conn(0x7fc9dc153780 legacy=0x7fc9dc0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9db7fe700 1 -- 172.21.15.5:0/1656672022 >> v1:172.21.15.5:6789/0 conn(0x7fc9dc153b50 legacy=0x7fc9dc0ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9db7fe700 1 -- 172.21.15.5:0/1656672022 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc15d280 con 0x7fc9dc0cd380 2023-06-23T01:04:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9d9ffb700 1 -- 172.21.15.5:0/1656672022 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729963984 0 0) 0x7fc9dc0afce0 con 0x7fc9dc153b50 2023-06-23T01:04:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.642+0000 7fc9db7fe700 1 --2- 172.21.15.5:0/1656672022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc15a4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009e60 tx=0x7fc9d4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433d99fef59d4c48 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9d9ffb700 1 -- 172.21.15.5:0/1656672022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400e070 con 0x7fc9dc0cd380 2023-06-23T01:04:41.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9d9ffb700 1 -- 172.21.15.5:0/1656672022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d400bcc0 con 0x7fc9dc0cd380 2023-06-23T01:04:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9d9ffb700 1 -- 172.21.15.5:0/1656672022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4012680 con 0x7fc9dc0cd380 2023-06-23T01:04:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9e17db700 1 -- 172.21.15.5:0/1656672022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 msgr2=0x7fc9dc15a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9e17db700 1 --2- 172.21.15.5:0/1656672022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc15a4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009e60 tx=0x7fc9d4009780 comp rx=0 tx=0).stop 2023-06-23T01:04:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9e17db700 1 -- 172.21.15.5:0/1656672022 shutdown_connections 2023-06-23T01:04:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9e17db700 1 --2- 172.21.15.5:0/1656672022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc15a4d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9e17db700 1 -- 172.21.15.5:0/1656672022 >> 172.21.15.5:0/1656672022 conn(0x7fc9dc070be0 msgr2=0x7fc9dc06d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9e17db700 1 -- 172.21.15.5:0/1656672022 shutdown_connections 2023-06-23T01:04:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.643+0000 7fc9e17db700 1 -- 172.21.15.5:0/1656672022 wait complete. 2023-06-23T01:04:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.644+0000 7fc9e17db700 1 Processor -- start 2023-06-23T01:04:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9e17db700 1 -- start start 2023-06-23T01:04:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9e17db700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9e17db700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9dc153780 0x7fc9dc156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9e17db700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9dc153b50 0x7fc9dc0c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9e17db700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0c8610 con 0x7fc9dc0cd380 2023-06-23T01:04:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9e17db700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0c9190 con 0x7fc9dc153780 2023-06-23T01:04:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9e17db700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0c8330 con 0x7fc9dc153b50 2023-06-23T01:04:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc0c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43828/0 (socket says 172.21.15.5:43828) 2023-06-23T01:04:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.645+0000 7fc9daffd700 1 -- 172.21.15.5:0/856317602 learned_addr learned my addr 172.21.15.5:0/856317602 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9daffd700 1 -- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9dc153b50 msgr2=0x7fc9dc0c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9db7fe700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9dc153b50 0x7fc9dc0c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9da7fc700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9dc153780 0x7fc9dc156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9daffd700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9dc153b50 0x7fc9dc0c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9daffd700 1 -- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9dc153780 msgr2=0x7fc9dc156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9daffd700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9dc153780 0x7fc9dc156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9daffd700 1 -- 172.21.15.5:0/856317602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4009a20 con 0x7fc9dc0cd380 2023-06-23T01:04:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9da7fc700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9dc153780 0x7fc9dc156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9daffd700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc0c7d10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c590 tx=0x7fc9cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9c2ffd700 1 -- 172.21.15.5:0/856317602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc011590 con 0x7fc9dc0cd380 2023-06-23T01:04:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9c2ffd700 1 -- 172.21.15.5:0/856317602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc0116f0 con 0x7fc9dc0cd380 2023-06-23T01:04:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc0c8b40 con 0x7fc9dc0cd380 2023-06-23T01:04:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9c2ffd700 1 -- 172.21.15.5:0/856317602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc011ab0 con 0x7fc9dc0cd380 2023-06-23T01:04:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.646+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc0cba30 con 0x7fc9dc0cd380 2023-06-23T01:04:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.647+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc04c1b0 con 0x7fc9dc0cd380 2023-06-23T01:04:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.648+0000 7fc9c2ffd700 1 -- 172.21.15.5:0/856317602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7fc9cc01b3b0 con 0x7fc9dc0cd380 2023-06-23T01:04:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.648+0000 7fc9c2ffd700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fc9c8066e00 0x7fc9c80692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.648+0000 7fc9c2ffd700 1 -- 172.21.15.5:0/856317602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fc9cc083630 con 0x7fc9dc0cd380 2023-06-23T01:04:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.648+0000 7fc9da7fc700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fc9c8066e00 0x7fc9c80692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.649+0000 7fc9da7fc700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fc9c8066e00 0x7fc9c80692b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0cb510 tx=0x7fc9d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.653+0000 7fc9c2ffd700 1 -- 172.21.15.5:0/856317602 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fc9cc050c40 con 0x7fc9dc0cd380 2023-06-23T01:04:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:41.856+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc9dc15d560 con 0x7fc9dc0cd380 2023-06-23T01:04:42.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.495+0000 7fc9c2ffd700 1 -- 172.21.15.5:0/856317602 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc9cc0170c0 con 0x7fc9dc0cd380 2023-06-23T01:04:42.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fc9c8066e00 msgr2=0x7fc9c80692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:42.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fc9c8066e00 0x7fc9c80692b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0cb510 tx=0x7fc9d0009040 comp rx=0 tx=0).stop 2023-06-23T01:04:42.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 msgr2=0x7fc9dc0c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:42.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc0c7d10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00c590 tx=0x7fc9cc00c950 comp rx=0 tx=0).stop 2023-06-23T01:04:42.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 shutdown_connections 2023-06-23T01:04:42.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fc9c8066e00 0x7fc9c80692b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc9dc153b50 0x7fc9dc0c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9dc0cd380 0x7fc9dc0c7d10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 --2- 172.21.15.5:0/856317602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9dc153780 0x7fc9dc156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.497+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 >> 172.21.15.5:0/856317602 conn(0x7fc9dc070be0 msgr2=0x7fc9dc0c5520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.498+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 shutdown_connections 2023-06-23T01:04:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.498+0000 7fc9e17db700 1 -- 172.21.15.5:0/856317602 wait complete. 2023-06-23T01:04:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-23T01:04:42.514 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-23T01:04:42.514 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-23T01:04:42.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-23T01:04:42.517 INFO:tasks.ceph.mds.a:Started 2023-06-23T01:04:42.517 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-23T01:04:42.517 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-23T01:04:42.519 INFO:tasks.ceph.mds.c:Started 2023-06-23T01:04:42.520 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-23T01:04:42.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-23T01:04:42.523 INFO:tasks.ceph.mds.b:Started 2023-06-23T01:04:42.523 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-23T01:04:42.523 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-23T01:04:42.525 INFO:tasks.ceph.mds.d:Started 2023-06-23T01:04:42.526 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-23T01:04:42.526 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:04:42.596 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2023-06-23T01:04:42.597 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2023-06-23T01:04:42.599 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2023-06-23T01:04:42.605 INFO:tasks.ceph.mds.d.smithi152.stdout:starting mds.d at 2023-06-23T01:04:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.654+0000 7f460c70a700 1 Processor -- start 2023-06-23T01:04:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.654+0000 7f460c70a700 1 -- start start 2023-06-23T01:04:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.655+0000 7f460c70a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f460406a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.655+0000 7f460c70a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f460406a860 0x7f46040ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.655+0000 7f460c70a700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46040cd850 con 0x7f46040cd450 2023-06-23T01:04:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.655+0000 7f460c70a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46040c7d00 con 0x7f460406a860 2023-06-23T01:04:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.655+0000 7f460c70a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460406ae90 con 0x7f4604155f60 2023-06-23T01:04:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.655+0000 7f460aca7700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f46040cd450 0x7f46040c9890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40166/0 (socket says 172.21.15.5:40166) 2023-06-23T01:04:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.655+0000 7f460aca7700 1 -- 172.21.15.5:0/295786466 learned_addr learned my addr 172.21.15.5:0/295786466 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f460a4a6700 1 --2- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f460406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f4609ca5700 1 --2- 172.21.15.5:0/295786466 >> v2:172.21.15.152:3300/0 conn(0x7f460406a860 0x7f46040ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f46094a4700 1 -- 172.21.15.5:0/295786466 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571641993 0 0) 0x7f46040cd850 con 0x7f46040cd450 2023-06-23T01:04:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f46094a4700 1 -- 172.21.15.5:0/295786466 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f0003570 con 0x7f46040cd450 2023-06-23T01:04:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f4609ca5700 1 -- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 msgr2=0x7f460406a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f4609ca5700 1 --2- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f460406a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f46094a4700 1 -- 172.21.15.5:0/295786466 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2729253075 0 0) 0x7f45f0003570 con 0x7f46040cd450 2023-06-23T01:04:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f4609ca5700 1 -- 172.21.15.5:0/295786466 >> v1:172.21.15.5:6789/0 conn(0x7f46040cd450 legacy=0x7f46040c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f4609ca5700 1 -- 172.21.15.5:0/295786466 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460406b170 con 0x7f460406a860 2023-06-23T01:04:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f460a4a6700 1 --2- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f460406a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f4609ca5700 1 --2- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460406a860 0x7f46040ccf10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f46000098e0 tx=0x7f46000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba92229335f245b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f46094a4700 1 -- 172.21.15.5:0/295786466 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0017f0 con 0x7f460406a860 2023-06-23T01:04:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f46094a4700 1 -- 172.21.15.5:0/295786466 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4600011420 con 0x7f460406a860 2023-06-23T01:04:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.656+0000 7f46094a4700 1 -- 172.21.15.5:0/295786466 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46000117c0 con 0x7f460406a860 2023-06-23T01:04:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.657+0000 7f460c70a700 1 -- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460406a860 msgr2=0x7f46040ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.657+0000 7f460c70a700 1 --2- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460406a860 0x7f46040ccf10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f46000098e0 tx=0x7f46000091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.657+0000 7f460c70a700 1 -- 172.21.15.5:0/295786466 shutdown_connections 2023-06-23T01:04:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.657+0000 7f460c70a700 1 --2- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f460406a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.657+0000 7f460c70a700 1 --2- 172.21.15.5:0/295786466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f460406a860 0x7f46040ccf10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.657+0000 7f460c70a700 1 -- 172.21.15.5:0/295786466 >> 172.21.15.5:0/295786466 conn(0x7f4604068240 msgr2=0x7f4604068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.657+0000 7f460c70a700 1 -- 172.21.15.5:0/295786466 shutdown_connections 2023-06-23T01:04:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.657+0000 7f460c70a700 1 -- 172.21.15.5:0/295786466 wait complete. 2023-06-23T01:04:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.658+0000 7f460c70a700 1 Processor -- start 2023-06-23T01:04:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.658+0000 7f460c70a700 1 -- start start 2023-06-23T01:04:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460c70a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f460406a860 0x7f46040aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460c70a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46040cd450 0x7f460414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460c70a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f46040b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460c70a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4604166790 con 0x7f460406a860 2023-06-23T01:04:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460c70a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46040af320 con 0x7f46040cd450 2023-06-23T01:04:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460c70a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46040c4680 con 0x7f4604155f60 2023-06-23T01:04:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f460406a860 0x7f46040aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f460406a860 0x7f46040aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43858/0 (socket says 172.21.15.5:43858) 2023-06-23T01:04:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 -- 172.21.15.5:0/4071601500 learned_addr learned my addr 172.21.15.5:0/4071601500 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f4609ca5700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46040cd450 0x7f460414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460aca7700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f46040b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 -- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 msgr2=0x7f46040b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f46040b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 -- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46040cd450 msgr2=0x7f460414fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46040cd450 0x7f460414fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 -- 172.21.15.5:0/4071601500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4009420 con 0x7f460406a860 2023-06-23T01:04:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f4609ca5700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46040cd450 0x7f460414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f460a4a6700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f460406a860 0x7f46040aede0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f45f400ec80 tx=0x7f45f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.659+0000 7f45fa7fc700 1 -- 172.21.15.5:0/4071601500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f400c9d0 con 0x7f460406a860 2023-06-23T01:04:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.660+0000 7f45fa7fc700 1 -- 172.21.15.5:0/4071601500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f400cb30 con 0x7f460406a860 2023-06-23T01:04:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.660+0000 7f45fa7fc700 1 -- 172.21.15.5:0/4071601500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f400ceb0 con 0x7f460406a860 2023-06-23T01:04:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.660+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4600009420 con 0x7f460406a860 2023-06-23T01:04:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.660+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4604166a20 con 0x7f460406a860 2023-06-23T01:04:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.661+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460404c1b0 con 0x7f460406a860 2023-06-23T01:04:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.661+0000 7f45fa7fc700 1 -- 172.21.15.5:0/4071601500 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 84592+0+0 (secure 0 0 0) 0x7f45f4013680 con 0x7f460406a860 2023-06-23T01:04:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.662+0000 7f45fa7fc700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f45e8066e00 0x7f45e80692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.662+0000 7f45fa7fc700 1 -- 172.21.15.5:0/4071601500 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f45f40845a0 con 0x7f460406a860 2023-06-23T01:04:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.662+0000 7f4609ca5700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f45e8066e00 0x7f45e80692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.663+0000 7f4609ca5700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f45e8066e00 0x7f45e80692b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f46040cd2d0 tx=0x7f4600016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:42.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.666+0000 7f45fa7fc700 1 -- 172.21.15.5:0/4071601500 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f45f4051ba0 con 0x7f460406a860 2023-06-23T01:04:42.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.720+0000 7f45fa7fc700 1 -- 172.21.15.5:0/4071601500 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f45f4051f70 con 0x7f460406a860 2023-06-23T01:04:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.867+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46040cd020 con 0x7f460406a860 2023-06-23T01:04:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.867+0000 7f45fa7fc700 1 -- 172.21.15.5:0/4071601500 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13612 (secure 0 0 0) 0x7f45f406e1c0 con 0x7f460406a860 2023-06-23T01:04:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:04:42.871 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"fsid":"002f2efe-a784-4dcf-91c2-b9f520cd2c9c","created":"2023-06-23T01:04:28.422097+0000","modified":"2023-06-23T01:04:42.488513+0000","last_up_change":"2023-06-23T01:04:39.254387+0000","last_in_change":"2023-06-23T01:04:31.676217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:04:40.075004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T01:04:40.525837+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2499edeb-6ba9-4392-b867-6a90f953982a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":87985},{"type":"v1","addr":"172.21.15.5:6817","nonce":87985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":87985},{"type":"v1","addr":"172.21.15.5:6819","nonce":87985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":87985},{"type":"v1","addr":"172.21.15.5:6826","nonce":87985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":87985},{"type":"v1","addr":"172.21.15.5:6822","nonce":87985}]},"public_addr":"172.21.15.5:6817/87985","cluster_addr":"172.21.15.5:6819/87985","heartbeat_back_addr":"172.21.15.5:6826/87985","heartbeat_front_addr":"172.21.15.5:6822/87985","state":["exists","up"]},{"osd":1,"uuid":"f2aa5a85-0aea-4bef-b45a-78297c1cd686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":87988},{"type":"v1","addr":"172.21.15.5:6801","nonce":87988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":87988},{"type":"v1","addr":"172.21.15.5:6804","nonce":87988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":87988},{"type":"v1","addr":"172.21.15.5:6812","nonce":87988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":87988},{"type":"v1","addr":"172.21.15.5:6808","nonce":87988}]},"public_addr":"172.21.15.5:6801/87988","cluster_addr":"172.21.15.5:6804/87988","heartbeat_back_addr":"172.21.15.5:6812/87988","heartbeat_front_addr":"172.21.15.5:6808/87988","state":["exists","up"]},{"osd":2,"uuid":"72aa25ff-c758-4780-9b74-fc1c316a3264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6821","nonce":87990},{"type":"v1","addr":"172.21.15.5:6823","nonce":87990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6825","nonce":87990},{"type":"v1","addr":"172.21.15.5:6827","nonce":87990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":87990},{"type":"v1","addr":"172.21.15.5:6831","nonce":87990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":87990},{"type":"v1","addr":"172.21.15.5:6829","nonce":87990}]},"public_addr":"172.21.15.5:6823/87990","cluster_addr":"172.21.15.5:6827/87990","heartbeat_back_addr":"172.21.15.5:6831/87990","heartbeat_front_addr":"172.21.15.5:6829/87990","state":["exists","up"]},{"osd":3,"uuid":"77dd6974-49b0-4288-b310-e7a0729ea2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":87991},{"type":"v1","addr":"172.21.15.5:6805","nonce":87991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":87991},{"type":"v1","addr":"172.21.15.5:6809","nonce":87991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":87991},{"type":"v1","addr":"172.21.15.5:6815","nonce":87991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":87991},{"type":"v1","addr":"172.21.15.5:6813","nonce":87991}]},"public_addr":"172.21.15.5:6805/87991","cluster_addr":"172.21.15.5:6809/87991","heartbeat_back_addr":"172.21.15.5:6815/87991","heartbeat_front_addr":"172.21.15.5:6813/87991","state":["exists","up"]},{"osd":4,"uuid":"80562fc5-676d-4cee-9978-4a94fca8df84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":88231},{"type":"v1","addr":"172.21.15.152:6817","nonce":88231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":88231},{"type":"v1","addr":"172.21.15.152:6819","nonce":88231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":88231},{"type":"v1","addr":"172.21.15.152:6823","nonce":88231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":88231},{"type":"v1","addr":"172.21.15.152:6821","nonce":88231}]},"public_addr":"172.21.15.152:6817/88231","cluster_addr":"172.21.15.152:6819/88231","heartbeat_back_addr":"172.21.15.152:6823/88231","heartbeat_front_addr":"172.21.15.152:6821/88231","state":["exists","up"]},{"osd":5,"uuid":"1db79ad4-997e-4516-a40b-90ee44d4957a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":88215},{"type":"v1","addr":"172.21.15.152:6801","nonce":88215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":88215},{"type":"v1","addr":"172.21.15.152:6803","nonce":88215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":88215},{"type":"v1","addr":"172.21.15.152:6807","nonce":88215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":88215},{"type":"v1","addr":"172.21.15.152:6805","nonce":88215}]},"public_addr":"172.21.15.152:6801/88215","cluster_addr":"172.21.15.152:6803/88215","heartbeat_back_addr":"172.21.15.152:6807/88215","heartbeat_front_addr":"172.21.15.152:6805/88215","state":["exists","up"]},{"osd":6,"uuid":"c18d6095-c8f0-4d76-94a8-f8de40a45195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":88216},{"type":"v1","addr":"172.21.15.152:6809","nonce":88216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":88216},{"type":"v1","addr":"172.21.15.152:6811","nonce":88216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":88216},{"type":"v1","addr":"172.21.15.152:6815","nonce":88216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":88216},{"type":"v1","addr":"172.21.15.152:6813","nonce":88216}]},"public_addr":"172.21.15.152:6809/88216","cluster_addr":"172.21.15.152:6811/88216","heartbeat_back_addr":"172.21.15.152:6815/88216","heartbeat_front_addr":"172.21.15.152:6813/88216","state":["exists","up"]},{"osd":7,"uuid":"6ade832e-0e46-4c4f-8fb2-3a86a643e19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":88246},{"type":"v1","addr":"172.21.15.152:6825","nonce":88246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":88246},{"type":"v1","addr":"172.21.15.152:6827","nonce":88246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":88246},{"type":"v1","addr":"172.21.15.152:6831","nonce":88246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":88246},{"type":"v1","addr":"172.21.15.152:6829","nonce":88246}]},"public_addr":"172.21.15.152:6825/88246","cluster_addr":"172.21.15.152:6827/88246","heartbeat_back_addr":"172.21.15.152:6831/88246","heartbeat_front_addr":"172.21.15.152:6829/88246","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.831143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.816429+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.901041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.813091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.819606+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.865581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.822019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.825405+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:04:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f45e8066e00 msgr2=0x7f45e80692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f45e8066e00 0x7f45e80692b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f46040cd2d0 tx=0x7f4600016000 comp rx=0 tx=0).stop 2023-06-23T01:04:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f460406a860 msgr2=0x7f46040aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f460406a860 0x7f46040aede0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f45f400ec80 tx=0x7f45f40091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 shutdown_connections 2023-06-23T01:04:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f45e8066e00 0x7f45e80692b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4604155f60 0x7f46040b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f460406a860 0x7f46040aede0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 --2- 172.21.15.5:0/4071601500 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46040cd450 0x7f460414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 >> 172.21.15.5:0/4071601500 conn(0x7f4604068240 msgr2=0x7f4604154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 shutdown_connections 2023-06-23T01:04:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:42.870+0000 7f460c70a700 1 -- 172.21.15.5:0/4071601500 wait complete. 2023-06-23T01:04:42.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T01:04:40.075004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T01:04:40.525837+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-06-23T01:04:42.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T01:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.005+0000 7fad30753700 1 Processor -- start 2023-06-23T01:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.005+0000 7fad30753700 1 -- start start 2023-06-23T01:04:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad30753700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad280b2550 0x7fad2815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad30753700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280cede0 0x7fad280ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad30753700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2815cf70 con 0x7fad280b2920 2023-06-23T01:04:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad30753700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280cb470 con 0x7fad280b2550 2023-06-23T01:04:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad30753700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280cb750 con 0x7fad280cede0 2023-06-23T01:04:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2e4ef700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad280b2550 0x7fad2815d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2ecf0700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fad280b2920 0x7fad2806ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40174/0 (socket says 172.21.15.5:40174) 2023-06-23T01:04:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2e4ef700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad280b2550 0x7fad2815d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43868/0 (socket says 172.21.15.5:43868) 2023-06-23T01:04:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2ecf0700 1 -- 172.21.15.5:0/4162203300 learned_addr learned my addr 172.21.15.5:0/4162203300 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2dcee700 1 --2- 172.21.15.5:0/4162203300 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280cede0 0x7fad280ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2e4ef700 1 -- 172.21.15.5:0/4162203300 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280cede0 msgr2=0x7fad280ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2e4ef700 1 --2- 172.21.15.5:0/4162203300 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280cede0 0x7fad280ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2e4ef700 1 -- 172.21.15.5:0/4162203300 >> v1:172.21.15.5:6789/0 conn(0x7fad280b2920 legacy=0x7fad2806ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2e4ef700 1 -- 172.21.15.5:0/4162203300 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2815d1c0 con 0x7fad280b2550 2023-06-23T01:04:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2d4ed700 1 -- 172.21.15.5:0/4162203300 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726475857 0 0) 0x7fad2815cf70 con 0x7fad280b2920 2023-06-23T01:04:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2e4ef700 1 --2- 172.21.15.5:0/4162203300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2550 0x7fad2815d9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad18009e60 tx=0x7fad18009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92d46fbb26d4a93 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2d4ed700 1 -- 172.21.15.5:0/4162203300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800e070 con 0x7fad280b2550 2023-06-23T01:04:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2d4ed700 1 -- 172.21.15.5:0/4162203300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1800bc30 con 0x7fad280b2550 2023-06-23T01:04:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.006+0000 7fad2d4ed700 1 -- 172.21.15.5:0/4162203300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18012430 con 0x7fad280b2550 2023-06-23T01:04:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.007+0000 7fad30753700 1 -- 172.21.15.5:0/4162203300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2550 msgr2=0x7fad2815d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.007+0000 7fad30753700 1 --2- 172.21.15.5:0/4162203300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2550 0x7fad2815d9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad18009e60 tx=0x7fad18009780 comp rx=0 tx=0).stop 2023-06-23T01:04:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.007+0000 7fad30753700 1 -- 172.21.15.5:0/4162203300 shutdown_connections 2023-06-23T01:04:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.007+0000 7fad30753700 1 --2- 172.21.15.5:0/4162203300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2550 0x7fad2815d9c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.007+0000 7fad30753700 1 --2- 172.21.15.5:0/4162203300 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280cede0 0x7fad280ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.007+0000 7fad30753700 1 -- 172.21.15.5:0/4162203300 >> 172.21.15.5:0/4162203300 conn(0x7fad280d02a0 msgr2=0x7fad280d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.007+0000 7fad30753700 1 -- 172.21.15.5:0/4162203300 shutdown_connections 2023-06-23T01:04:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.007+0000 7fad30753700 1 -- 172.21.15.5:0/4162203300 wait complete. 2023-06-23T01:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.008+0000 7fad30753700 1 Processor -- start 2023-06-23T01:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.008+0000 7fad30753700 1 -- start start 2023-06-23T01:04:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad30753700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280b2550 0x7fad28152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad30753700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2920 0x7fad280b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad30753700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad280cede0 0x7fad28145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad30753700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2815ea60 con 0x7fad280b2920 2023-06-23T01:04:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad30753700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281571b0 con 0x7fad280cede0 2023-06-23T01:04:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad30753700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280ca180 con 0x7fad280b2550 2023-06-23T01:04:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2920 0x7fad280b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2920 0x7fad280b3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43882/0 (socket says 172.21.15.5:43882) 2023-06-23T01:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 -- 172.21.15.5:0/2729268271 learned_addr learned my addr 172.21.15.5:0/2729268271 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2e4ef700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280b2550 0x7fad28152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 -- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280b2550 msgr2=0x7fad28152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2ecf0700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad280cede0 0x7fad28145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280b2550 0x7fad28152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 -- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad280cede0 msgr2=0x7fad28145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad280cede0 0x7fad28145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 -- 172.21.15.5:0/2729268271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad18009a20 con 0x7fad280b2920 2023-06-23T01:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2e4ef700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280b2550 0x7fad28152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2ecf0700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad280cede0 0x7fad28145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.009+0000 7fad2dcee700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2920 0x7fad280b3040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d890 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.010+0000 7fad267fc700 1 -- 172.21.15.5:0/2729268271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00b600 con 0x7fad280b2920 2023-06-23T01:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.010+0000 7fad267fc700 1 -- 172.21.15.5:0/2729268271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c00b760 con 0x7fad280b2920 2023-06-23T01:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.010+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad28156ef0 con 0x7fad280b2920 2023-06-23T01:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.010+0000 7fad267fc700 1 -- 172.21.15.5:0/2729268271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00bae0 con 0x7fad280b2920 2023-06-23T01:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.010+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad281557e0 con 0x7fad280b2920 2023-06-23T01:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.011+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2815ebb0 con 0x7fad280b2920 2023-06-23T01:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.011+0000 7fad267fc700 1 -- 172.21.15.5:0/2729268271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fad1c01e460 con 0x7fad280b2920 2023-06-23T01:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.011+0000 7fad267fc700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fad14066e90 0x7fad14069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.012+0000 7fad267fc700 1 -- 172.21.15.5:0/2729268271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fad1c0848e0 con 0x7fad280b2920 2023-06-23T01:04:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.012+0000 7fad2e4ef700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fad14066e90 0x7fad14069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.012+0000 7fad2e4ef700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fad14066e90 0x7fad14069340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fad18009e60 tx=0x7fad18019000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.016+0000 7fad267fc700 1 -- 172.21.15.5:0/2729268271 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fad2815ebb0 con 0x7fad280b2920 2023-06-23T01:04:43.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.210+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad2815ebb0 con 0x7fad280b2920 2023-06-23T01:04:43.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.211+0000 7fad267fc700 1 -- 172.21.15.5:0/2729268271 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad2815ebb0 con 0x7fad280b2920 2023-06-23T01:04:43.214 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-06-23T01:04:43.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.212+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fad14066e90 msgr2=0x7fad14069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.212+0000 7fad30753700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fad14066e90 0x7fad14069340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fad18009e60 tx=0x7fad18019000 comp rx=0 tx=0).stop 2023-06-23T01:04:43.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.212+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2920 msgr2=0x7fad280b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.212+0000 7fad30753700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2920 0x7fad280b3040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d890 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2023-06-23T01:04:43.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.212+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 shutdown_connections 2023-06-23T01:04:43.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.213+0000 7fad30753700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fad14066e90 0x7fad14069340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.213+0000 7fad30753700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad280cede0 0x7fad28145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.213+0000 7fad30753700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fad280b2550 0x7fad28152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.213+0000 7fad30753700 1 --2- 172.21.15.5:0/2729268271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad280b2920 0x7fad280b3040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.213+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 >> 172.21.15.5:0/2729268271 conn(0x7fad280d02a0 msgr2=0x7fad280d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:43.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.213+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 shutdown_connections 2023-06-23T01:04:43.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.213+0000 7fad30753700 1 -- 172.21.15.5:0/2729268271 wait complete. 2023-06-23T01:04:43.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T01:04:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.346+0000 7f759e82a700 1 Processor -- start 2023-06-23T01:04:43.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.346+0000 7f759e82a700 1 -- start start 2023-06-23T01:04:43.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759e82a700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7598158f20 0x7f75980b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759e82a700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75980cc560 0x7f75981592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759e82a700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759815dda0 con 0x7f759815ceb0 2023-06-23T01:04:43.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759e82a700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7598159830 con 0x7f75980cc560 2023-06-23T01:04:43.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759e82a700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75980cb100 con 0x7f7598158f20 2023-06-23T01:04:43.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d027700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75980cc560 0x7f75981592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d027700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75980cc560 0x7f75981592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43890/0 (socket says 172.21.15.5:43890) 2023-06-23T01:04:43.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d027700 1 -- 172.21.15.5:0/1528200162 learned_addr learned my addr 172.21.15.5:0/1528200162 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:43.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d828700 1 --2- 172.21.15.5:0/1528200162 >> v2:172.21.15.152:3300/0 conn(0x7f7598158f20 0x7f75980b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d027700 1 -- 172.21.15.5:0/1528200162 >> v2:172.21.15.152:3300/0 conn(0x7f7598158f20 msgr2=0x7f75980b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:43.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d027700 1 --2- 172.21.15.5:0/1528200162 >> v2:172.21.15.152:3300/0 conn(0x7f7598158f20 0x7f75980b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d027700 1 -- 172.21.15.5:0/1528200162 >> v1:172.21.15.152:6789/0 conn(0x7f759815ceb0 legacy=0x7f759806eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d027700 1 -- 172.21.15.5:0/1528200162 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759815d7e0 con 0x7f75980cc560 2023-06-23T01:04:43.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.347+0000 7f759d027700 1 --2- 172.21.15.5:0/1528200162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75981592f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7598071030 tx=0x7f7588009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441d3ec927964d91 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759c826700 1 -- 172.21.15.5:0/1528200162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588009e70 con 0x7f75980cc560 2023-06-23T01:04:43.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759c826700 1 -- 172.21.15.5:0/1528200162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758800b860 con 0x7f75980cc560 2023-06-23T01:04:43.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759c826700 1 -- 172.21.15.5:0/1528200162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800bc00 con 0x7f75980cc560 2023-06-23T01:04:43.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759e82a700 1 -- 172.21.15.5:0/1528200162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 msgr2=0x7f75981592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759e82a700 1 --2- 172.21.15.5:0/1528200162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75981592f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7598071030 tx=0x7f7588009250 comp rx=0 tx=0).stop 2023-06-23T01:04:43.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759e82a700 1 -- 172.21.15.5:0/1528200162 shutdown_connections 2023-06-23T01:04:43.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759e82a700 1 --2- 172.21.15.5:0/1528200162 >> v2:172.21.15.152:3300/0 conn(0x7f7598158f20 0x7f75980b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759e82a700 1 --2- 172.21.15.5:0/1528200162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75981592f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759e82a700 1 -- 172.21.15.5:0/1528200162 >> 172.21.15.5:0/1528200162 conn(0x7f7598070c10 msgr2=0x7f75980d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:43.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759e82a700 1 -- 172.21.15.5:0/1528200162 shutdown_connections 2023-06-23T01:04:43.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.348+0000 7f759e82a700 1 -- 172.21.15.5:0/1528200162 wait complete. 2023-06-23T01:04:43.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.349+0000 7f759e82a700 1 Processor -- start 2023-06-23T01:04:43.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.349+0000 7f759e82a700 1 -- start start 2023-06-23T01:04:43.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759e82a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75980c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759e82a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7598158f20 0x7f75980c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759e82a700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f759815ceb0 0x7f75980c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759e82a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7598159cc0 con 0x7f75980cc560 2023-06-23T01:04:43.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759e82a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75980c4b70 con 0x7f7598158f20 2023-06-23T01:04:43.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759e82a700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75980c4e50 con 0x7f759815ceb0 2023-06-23T01:04:43.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d828700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75980c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d828700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75980c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43906/0 (socket says 172.21.15.5:43906) 2023-06-23T01:04:43.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d828700 1 -- 172.21.15.5:0/2690539413 learned_addr learned my addr 172.21.15.5:0/2690539413 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:43.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d027700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7598158f20 0x7f75980c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d828700 1 -- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f759815ceb0 msgr2=0x7f75980c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759e029700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f759815ceb0 0x7f75980c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d828700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f759815ceb0 0x7f75980c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d828700 1 -- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7598158f20 msgr2=0x7f75980c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d828700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7598158f20 0x7f75980c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.350+0000 7f759d828700 1 -- 172.21.15.5:0/2690539413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880094f0 con 0x7f75980cc560 2023-06-23T01:04:43.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.351+0000 7f759d828700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75980c9cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f759400ec80 tx=0x7f75940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.351+0000 7f758dffb700 1 -- 172.21.15.5:0/2690539413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400c9d0 con 0x7f75980cc560 2023-06-23T01:04:43.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.351+0000 7f758dffb700 1 -- 172.21.15.5:0/2690539413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759400cb30 con 0x7f75980cc560 2023-06-23T01:04:43.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.351+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759815abb0 con 0x7f75980cc560 2023-06-23T01:04:43.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.351+0000 7f758dffb700 1 -- 172.21.15.5:0/2690539413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594013470 con 0x7f75980cc560 2023-06-23T01:04:43.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.351+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75980c5130 con 0x7f75980cc560 2023-06-23T01:04:43.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.352+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759804c1b0 con 0x7f75980cc560 2023-06-23T01:04:43.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.352+0000 7f758dffb700 1 -- 172.21.15.5:0/2690539413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f7594013820 con 0x7f75980cc560 2023-06-23T01:04:43.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.353+0000 7f758dffb700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f757c066e90 0x7f757c069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.353+0000 7f758dffb700 1 -- 172.21.15.5:0/2690539413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f75940845b0 con 0x7f75980cc560 2023-06-23T01:04:43.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.353+0000 7f759d027700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f757c066e90 0x7f757c069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.353+0000 7f759d027700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f757c066e90 0x7f757c069340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7598071030 tx=0x7f7588017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.357+0000 7f758dffb700 1 -- 172.21.15.5:0/2690539413 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f7594051ad0 con 0x7f75980cc560 2023-06-23T01:04:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.552+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f75980ca6d0 con 0x7f75980cc560 2023-06-23T01:04:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.552+0000 7f758dffb700 1 -- 172.21.15.5:0/2690539413 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f75940550f0 con 0x7f75980cc560 2023-06-23T01:04:43.556 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 8 2023-06-23T01:04:43.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f757c066e90 msgr2=0x7f757c069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f757c066e90 0x7f757c069340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7598071030 tx=0x7f7588017000 comp rx=0 tx=0).stop 2023-06-23T01:04:43.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 msgr2=0x7f75980c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75980c9cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f759400ec80 tx=0x7f75940091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 shutdown_connections 2023-06-23T01:04:43.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f757c066e90 0x7f757c069340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f759815ceb0 0x7f75980c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75980cc560 0x7f75980c9cf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 --2- 172.21.15.5:0/2690539413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7598158f20 0x7f75980c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 >> 172.21.15.5:0/2690539413 conn(0x7f7598070c10 msgr2=0x7f75980ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 shutdown_connections 2023-06-23T01:04:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.554+0000 7f759e82a700 1 -- 172.21.15.5:0/2690539413 wait complete. 2023-06-23T01:04:43.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T01:04:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.688+0000 7f62d421c700 1 Processor -- start 2023-06-23T01:04:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.688+0000 7f62d421c700 1 -- start start 2023-06-23T01:04:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.689+0000 7f62d421c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc156e10 0x7f62cc0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.689+0000 7f62d421c700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc06b380 con 0x7f62cc0af5f0 2023-06-23T01:04:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.689+0000 7f62d421c700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc15bc40 con 0x7f62cc0ccf10 2023-06-23T01:04:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.689+0000 7f62d421c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc15ad90 con 0x7f62cc156e10 2023-06-23T01:04:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.689+0000 7f62d27b9700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f62cc0af5f0 0x7f62cc06ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40176/0 (socket says 172.21.15.5:40176) 2023-06-23T01:04:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.689+0000 7f62d27b9700 1 -- 172.21.15.5:0/1173011294 learned_addr learned my addr 172.21.15.5:0/1173011294 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d1fb8700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc156e10 0x7f62cc0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442703355 0 0) 0x7f62cc06b380 con 0x7f62cc0af5f0 2023-06-23T01:04:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62b4003570 con 0x7f62cc0af5f0 2023-06-23T01:04:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3850714765 0 0) 0x7f62b4003570 con 0x7f62cc0af5f0 2023-06-23T01:04:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc156e10 msgr2=0x7f62cc0ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc156e10 0x7f62cc0ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 >> v1:172.21.15.152:6789/0 conn(0x7f62cc0ccf10 legacy=0x7f62cc06a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc15b030 con 0x7f62cc0af5f0 2023-06-23T01:04:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1394223029 0 0) 0x7f62bc001470 con 0x7f62cc0af5f0 2023-06-23T01:04:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 >> v1:172.21.15.5:6789/0 conn(0x7f62cc0af5f0 legacy=0x7f62cc06ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc156e10 0x7f62b4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4003fd0 0x7f62b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62b4004d50 0x7f62b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc15ad90 con 0x7f62b4004d50 2023-06-23T01:04:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b400bb40 con 0x7f62b4003fd0 2023-06-23T01:04:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b400bd00 con 0x7f62cc156e10 2023-06-23T01:04:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d27b9700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62b4004d50 0x7f62b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d1fb8700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc156e10 0x7f62b4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d17b7700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4003fd0 0x7f62b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d27b9700 1 -- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc156e10 msgr2=0x7f62b4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d27b9700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc156e10 0x7f62b4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d27b9700 1 -- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4003fd0 msgr2=0x7f62b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d27b9700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62b4003fd0 0x7f62b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.690+0000 7f62d27b9700 1 -- 172.21.15.5:0/1173011294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b4003570 con 0x7f62b4004d50 2023-06-23T01:04:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d27b9700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62b4004d50 0x7f62b40051c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00c390 tx=0x7f62cc15bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceadd815b3d6db34 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00cc60 con 0x7f62b4004d50 2023-06-23T01:04:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc00ce00 con 0x7f62b4004d50 2023-06-23T01:04:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d0fb6700 1 -- 172.21.15.5:0/1173011294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc005b70 con 0x7f62b4004d50 2023-06-23T01:04:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d421c700 1 -- 172.21.15.5:0/1173011294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62b4004d50 msgr2=0x7f62b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d421c700 1 --2- 172.21.15.5:0/1173011294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62b4004d50 0x7f62b40051c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00c390 tx=0x7f62cc15bbb0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d27b9700 1 -- 172.21.15.5:0/1173011294 reap_dead start 2023-06-23T01:04:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d421c700 1 -- 172.21.15.5:0/1173011294 shutdown_connections 2023-06-23T01:04:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d421c700 1 -- 172.21.15.5:0/1173011294 >> 172.21.15.5:0/1173011294 conn(0x7f62cc068240 msgr2=0x7f62cc068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d421c700 1 -- 172.21.15.5:0/1173011294 shutdown_connections 2023-06-23T01:04:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.691+0000 7f62d421c700 1 -- 172.21.15.5:0/1173011294 wait complete. 2023-06-23T01:04:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.692+0000 7f62d421c700 1 Processor -- start 2023-06-23T01:04:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d421c700 1 -- start start 2023-06-23T01:04:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d421c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc14a2f0 0x7f62cc0c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d421c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc14a6c0 0x7f62cc0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d421c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0c1790 0x7f62cc155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d421c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0b0a50 con 0x7f62cc0c1790 2023-06-23T01:04:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d421c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0c79a0 con 0x7f62cc14a6c0 2023-06-23T01:04:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d421c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0c2650 con 0x7f62cc14a2f0 2023-06-23T01:04:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d27b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0c1790 0x7f62cc155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d27b9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0c1790 0x7f62cc155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43924/0 (socket says 172.21.15.5:43924) 2023-06-23T01:04:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d27b9700 1 -- 172.21.15.5:0/3785473994 learned_addr learned my addr 172.21.15.5:0/3785473994 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d1fb8700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc14a2f0 0x7f62cc0c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.693+0000 7f62d17b7700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc14a6c0 0x7f62cc0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62d27b9700 1 -- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc14a2f0 msgr2=0x7f62cc0c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62d27b9700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc14a2f0 0x7f62cc0c1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62d27b9700 1 -- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc14a6c0 msgr2=0x7f62cc0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62d27b9700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc14a6c0 0x7f62cc0b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62d27b9700 1 -- 172.21.15.5:0/3785473994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0c2930 con 0x7f62cc0c1790 2023-06-23T01:04:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62d27b9700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0c1790 0x7f62cc155510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62bc011000 tx=0x7f62bc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62c9ffb700 1 -- 172.21.15.5:0/3785473994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc021070 con 0x7f62cc0c1790 2023-06-23T01:04:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62c9ffb700 1 -- 172.21.15.5:0/3785473994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc00cc60 con 0x7f62cc0c1790 2023-06-23T01:04:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc0c7630 con 0x7f62cc0c1790 2023-06-23T01:04:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62c9ffb700 1 -- 172.21.15.5:0/3785473994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc007320 con 0x7f62cc0c1790 2023-06-23T01:04:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.694+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc0c1be0 con 0x7f62cc0c1790 2023-06-23T01:04:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.695+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc06ca50 con 0x7f62cc0c1790 2023-06-23T01:04:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.695+0000 7f62c9ffb700 1 -- 172.21.15.5:0/3785473994 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f62bc007710 con 0x7f62cc0c1790 2023-06-23T01:04:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.696+0000 7f62c9ffb700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f62b8066e90 0x7f62b8069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.696+0000 7f62c9ffb700 1 -- 172.21.15.5:0/3785473994 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f62bc0848f0 con 0x7f62cc0c1790 2023-06-23T01:04:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.696+0000 7f62d1fb8700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f62b8066e90 0x7f62b8069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.696+0000 7f62d1fb8700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f62b8066e90 0x7f62b8069340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62c40094d0 tx=0x7f62c400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.700+0000 7f62c9ffb700 1 -- 172.21.15.5:0/3785473994 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f62bc051e10 con 0x7f62cc0c1790 2023-06-23T01:04:43.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.942+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62cc04c1b0 con 0x7f62cc0c1790 2023-06-23T01:04:43.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.942+0000 7f62c9ffb700 1 -- 172.21.15.5:0/3785473994 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f62bc017090 con 0x7f62cc0c1790 2023-06-23T01:04:43.947 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:04:43.947 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/3552295836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":3552295836},{"type":"v1","addr":"172.21.15.152:6837","nonce":3552295836}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4269,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3162139024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3162139024},{"type":"v1","addr":"172.21.15.5:6835","nonce":3162139024}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4297,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2474144562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2474144562},{"type":"v1","addr":"172.21.15.5:6834","nonce":2474144562}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2521352793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2521352793},{"type":"v1","addr":"172.21.15.152:6835","nonce":2521352793}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-23T01:04:43.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f62b8066e90 msgr2=0x7f62b8069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f62b8066e90 0x7f62b8069340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f62c40094d0 tx=0x7f62c400b040 comp rx=0 tx=0).stop 2023-06-23T01:04:43.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0c1790 msgr2=0x7f62cc155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:43.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0c1790 0x7f62cc155510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f62bc011000 tx=0x7f62bc015000 comp rx=0 tx=0).stop 2023-06-23T01:04:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 shutdown_connections 2023-06-23T01:04:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f62b8066e90 0x7f62b8069340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62cc0c1790 0x7f62cc155510 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f62cc14a2f0 0x7f62cc0c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 --2- 172.21.15.5:0/3785473994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc14a6c0 0x7f62cc0b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 >> 172.21.15.5:0/3785473994 conn(0x7f62cc068240 msgr2=0x7f62cc157830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 shutdown_connections 2023-06-23T01:04:43.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:43.945+0000 7f62d421c700 1 -- 172.21.15.5:0/3785473994 wait complete. 2023-06-23T01:04:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2023-06-23T01:04:43.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:04:44.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.076+0000 7fd8ec85b700 1 Processor -- start 2023-06-23T01:04:44.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.076+0000 7fd8ec85b700 1 -- start start 2023-06-23T01:04:44.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8ec85b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8e80cede0 0x7fd8e815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8ec85b700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e815cf70 con 0x7fd8e80b2920 2023-06-23T01:04:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8ec85b700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e80cb470 con 0x7fd8e80b2550 2023-06-23T01:04:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8ec85b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e80cb750 con 0x7fd8e80cede0 2023-06-23T01:04:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e5d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8e80cede0 0x7fd8e815d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e6d9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd8e80b2920 0x7fd8e806ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40192/0 (socket says 172.21.15.5:40192) 2023-06-23T01:04:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e6d9d700 1 -- 172.21.15.5:0/4254911289 learned_addr learned my addr 172.21.15.5:0/4254911289 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e5d9b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8e80cede0 0x7fd8e815d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43936/0 (socket says 172.21.15.5:43936) 2023-06-23T01:04:44.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e5d9b700 1 -- 172.21.15.5:0/4254911289 >> v1:172.21.15.152:6789/0 conn(0x7fd8e80b2550 legacy=0x7fd8e80ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e5d9b700 1 -- 172.21.15.5:0/4254911289 >> v1:172.21.15.5:6789/0 conn(0x7fd8e80b2920 legacy=0x7fd8e806ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e5d9b700 1 -- 172.21.15.5:0/4254911289 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e815d1c0 con 0x7fd8e80cede0 2023-06-23T01:04:44.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e559a700 1 -- 172.21.15.5:0/4254911289 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784194937 0 0) 0x7fd8e815cf70 con 0x7fd8e80b2920 2023-06-23T01:04:44.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e5d9b700 1 --2- 172.21.15.5:0/4254911289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e815d9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00098e0 tx=0x7fd8d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3529d5dbca0c3a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e559a700 1 -- 172.21.15.5:0/4254911289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0009e80 con 0x7fd8e80cede0 2023-06-23T01:04:44.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e559a700 1 -- 172.21.15.5:0/4254911289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d000b860 con 0x7fd8e80cede0 2023-06-23T01:04:44.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.077+0000 7fd8e559a700 1 -- 172.21.15.5:0/4254911289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d000bbe0 con 0x7fd8e80cede0 2023-06-23T01:04:44.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.078+0000 7fd8ec85b700 1 -- 172.21.15.5:0/4254911289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 msgr2=0x7fd8e815d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.078+0000 7fd8ec85b700 1 --2- 172.21.15.5:0/4254911289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e815d9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00098e0 tx=0x7fd8d00091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.078+0000 7fd8ec85b700 1 -- 172.21.15.5:0/4254911289 shutdown_connections 2023-06-23T01:04:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.078+0000 7fd8ec85b700 1 --2- 172.21.15.5:0/4254911289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e815d9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.078+0000 7fd8ec85b700 1 -- 172.21.15.5:0/4254911289 >> 172.21.15.5:0/4254911289 conn(0x7fd8e80d02a0 msgr2=0x7fd8e80d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:44.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.078+0000 7fd8ec85b700 1 -- 172.21.15.5:0/4254911289 shutdown_connections 2023-06-23T01:04:44.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.078+0000 7fd8ec85b700 1 -- 172.21.15.5:0/4254911289 wait complete. 2023-06-23T01:04:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.079+0000 7fd8ec85b700 1 Processor -- start 2023-06-23T01:04:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.079+0000 7fd8ec85b700 1 -- start start 2023-06-23T01:04:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8ec85b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8e80b2550 0x7fd8e814bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8ec85b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8e80b2920 0x7fd8e80b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8ec85b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e80c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8ec85b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c7f90 con 0x7fd8e80cede0 2023-06-23T01:04:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8ec85b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c73f0 con 0x7fd8e80b2550 2023-06-23T01:04:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8ec85b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c7660 con 0x7fd8e80b2920 2023-06-23T01:04:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e80c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e80c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43940/0 (socket says 172.21.15.5:43940) 2023-06-23T01:04:44.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 -- 172.21.15.5:0/2462931577 learned_addr learned my addr 172.21.15.5:0/2462931577 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:44.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e5d9b700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8e80b2920 0x7fd8e80b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e659c700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8e80b2550 0x7fd8e814bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 -- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8e80b2920 msgr2=0x7fd8e80b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8e80b2920 0x7fd8e80b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 -- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8e80b2550 msgr2=0x7fd8e814bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8e80b2550 0x7fd8e814bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 -- 172.21.15.5:0/2462931577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d0009420 con 0x7fd8e80cede0 2023-06-23T01:04:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.080+0000 7fd8e6d9d700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e80c6010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000c520 tx=0x7fd8e0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.081+0000 7fd8de7fc700 1 -- 172.21.15.5:0/2462931577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000ce90 con 0x7fd8e80cede0 2023-06-23T01:04:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.081+0000 7fd8de7fc700 1 -- 172.21.15.5:0/2462931577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e0005b30 con 0x7fd8e80cede0 2023-06-23T01:04:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.081+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e80c7940 con 0x7fd8e80cede0 2023-06-23T01:04:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.081+0000 7fd8de7fc700 1 -- 172.21.15.5:0/2462931577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00070d0 con 0x7fd8e80cede0 2023-06-23T01:04:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.081+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e814c2c0 con 0x7fd8e80cede0 2023-06-23T01:04:44.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.082+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e80c8100 con 0x7fd8e80cede0 2023-06-23T01:04:44.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.082+0000 7fd8de7fc700 1 -- 172.21.15.5:0/2462931577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fd8e0007500 con 0x7fd8e80cede0 2023-06-23T01:04:44.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.082+0000 7fd8de7fc700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fd8c8066ec0 0x7fd8c8069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.082+0000 7fd8de7fc700 1 -- 172.21.15.5:0/2462931577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd8e0083670 con 0x7fd8e80cede0 2023-06-23T01:04:44.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.083+0000 7fd8e659c700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fd8c8066ec0 0x7fd8c8069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.083+0000 7fd8e659c700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fd8c8066ec0 0x7fd8c8069370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8007fd0 tx=0x7fd8d8001650 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.087+0000 7fd8de7fc700 1 -- 172.21.15.5:0/2462931577 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fd8e0050b00 con 0x7fd8e80cede0 2023-06-23T01:04:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.279+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8e804c1b0 con 0x7fd8e80cede0 2023-06-23T01:04:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.280+0000 7fd8de7fc700 1 -- 172.21.15.5:0/2462931577 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13599 (secure 0 0 0) 0x7fd8e0017090 con 0x7fd8e80cede0 2023-06-23T01:04:44.284 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:04:44.284 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"fsid":"002f2efe-a784-4dcf-91c2-b9f520cd2c9c","created":"2023-06-23T01:04:28.422097+0000","modified":"2023-06-23T01:04:43.495352+0000","last_up_change":"2023-06-23T01:04:39.254387+0000","last_in_change":"2023-06-23T01:04:31.676217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:04:40.075004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T01:04:40.525837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"2499edeb-6ba9-4392-b867-6a90f953982a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":87985},{"type":"v1","addr":"172.21.15.5:6817","nonce":87985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":87985},{"type":"v1","addr":"172.21.15.5:6819","nonce":87985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":87985},{"type":"v1","addr":"172.21.15.5:6826","nonce":87985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":87985},{"type":"v1","addr":"172.21.15.5:6822","nonce":87985}]},"public_addr":"172.21.15.5:6817/87985","cluster_addr":"172.21.15.5:6819/87985","heartbeat_back_addr":"172.21.15.5:6826/87985","heartbeat_front_addr":"172.21.15.5:6822/87985","state":["exists","up"]},{"osd":1,"uuid":"f2aa5a85-0aea-4bef-b45a-78297c1cd686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":87988},{"type":"v1","addr":"172.21.15.5:6801","nonce":87988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":87988},{"type":"v1","addr":"172.21.15.5:6804","nonce":87988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":87988},{"type":"v1","addr":"172.21.15.5:6812","nonce":87988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":87988},{"type":"v1","addr":"172.21.15.5:6808","nonce":87988}]},"public_addr":"172.21.15.5:6801/87988","cluster_addr":"172.21.15.5:6804/87988","heartbeat_back_addr":"172.21.15.5:6812/87988","heartbeat_front_addr":"172.21.15.5:6808/87988","state":["exists","up"]},{"osd":2,"uuid":"72aa25ff-c758-4780-9b74-fc1c316a3264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6821","nonce":87990},{"type":"v1","addr":"172.21.15.5:6823","nonce":87990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6825","nonce":87990},{"type":"v1","addr":"172.21.15.5:6827","nonce":87990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":87990},{"type":"v1","addr":"172.21.15.5:6831","nonce":87990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":87990},{"type":"v1","addr":"172.21.15.5:6829","nonce":87990}]},"public_addr":"172.21.15.5:6823/87990","cluster_addr":"172.21.15.5:6827/87990","heartbeat_back_addr":"172.21.15.5:6831/87990","heartbeat_front_addr":"172.21.15.5:6829/87990","state":["exists","up"]},{"osd":3,"uuid":"77dd6974-49b0-4288-b310-e7a0729ea2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":87991},{"type":"v1","addr":"172.21.15.5:6805","nonce":87991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":87991},{"type":"v1","addr":"172.21.15.5:6809","nonce":87991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":87991},{"type":"v1","addr":"172.21.15.5:6815","nonce":87991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":87991},{"type":"v1","addr":"172.21.15.5:6813","nonce":87991}]},"public_addr":"172.21.15.5:6805/87991","cluster_addr":"172.21.15.5:6809/87991","heartbeat_back_addr":"172.21.15.5:6815/87991","heartbeat_front_addr":"172.21.15.5:6813/87991","state":["exists","up"]},{"osd":4,"uuid":"80562fc5-676d-4cee-9978-4a94fca8df84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":88231},{"type":"v1","addr":"172.21.15.152:6817","nonce":88231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":88231},{"type":"v1","addr":"172.21.15.152:6819","nonce":88231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":88231},{"type":"v1","addr":"172.21.15.152:6823","nonce":88231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":88231},{"type":"v1","addr":"172.21.15.152:6821","nonce":88231}]},"public_addr":"172.21.15.152:6817/88231","cluster_addr":"172.21.15.152:6819/88231","heartbeat_back_addr":"172.21.15.152:6823/88231","heartbeat_front_addr":"172.21.15.152:6821/88231","state":["exists","up"]},{"osd":5,"uuid":"1db79ad4-997e-4516-a40b-90ee44d4957a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":88215},{"type":"v1","addr":"172.21.15.152:6801","nonce":88215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":88215},{"type":"v1","addr":"172.21.15.152:6803","nonce":88215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":88215},{"type":"v1","addr":"172.21.15.152:6807","nonce":88215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":88215},{"type":"v1","addr":"172.21.15.152:6805","nonce":88215}]},"public_addr":"172.21.15.152:6801/88215","cluster_addr":"172.21.15.152:6803/88215","heartbeat_back_addr":"172.21.15.152:6807/88215","heartbeat_front_addr":"172.21.15.152:6805/88215","state":["exists","up"]},{"osd":6,"uuid":"c18d6095-c8f0-4d76-94a8-f8de40a45195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":88216},{"type":"v1","addr":"172.21.15.152:6809","nonce":88216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":88216},{"type":"v1","addr":"172.21.15.152:6811","nonce":88216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":88216},{"type":"v1","addr":"172.21.15.152:6815","nonce":88216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":88216},{"type":"v1","addr":"172.21.15.152:6813","nonce":88216}]},"public_addr":"172.21.15.152:6809/88216","cluster_addr":"172.21.15.152:6811/88216","heartbeat_back_addr":"172.21.15.152:6815/88216","heartbeat_front_addr":"172.21.15.152:6813/88216","state":["exists","up"]},{"osd":7,"uuid":"6ade832e-0e46-4c4f-8fb2-3a86a643e19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":88246},{"type":"v1","addr":"172.21.15.152:6825","nonce":88246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":88246},{"type":"v1","addr":"172.21.15.152:6827","nonce":88246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":88246},{"type":"v1","addr":"172.21.15.152:6831","nonce":88246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":88246},{"type":"v1","addr":"172.21.15.152:6829","nonce":88246}]},"public_addr":"172.21.15.152:6825/88246","cluster_addr":"172.21.15.152:6827/88246","heartbeat_back_addr":"172.21.15.152:6831/88246","heartbeat_front_addr":"172.21.15.152:6829/88246","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.831143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.816429+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.901041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.813091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.819606+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.865581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.822019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.825405+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:04:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fd8c8066ec0 msgr2=0x7fd8c8069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fd8c8066ec0 0x7fd8c8069370 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8007fd0 tx=0x7fd8d8001650 comp rx=0 tx=0).stop 2023-06-23T01:04:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 msgr2=0x7fd8e80c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e80c6010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000c520 tx=0x7fd8e0007f50 comp rx=0 tx=0).stop 2023-06-23T01:04:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 shutdown_connections 2023-06-23T01:04:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fd8c8066ec0 0x7fd8c8069370 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8e80cede0 0x7fd8e80c6010 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8e80b2550 0x7fd8e814bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 --2- 172.21.15.5:0/2462931577 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8e80b2920 0x7fd8e80b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 >> 172.21.15.5:0/2462931577 conn(0x7fd8e80d02a0 msgr2=0x7fd8e8157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 shutdown_connections 2023-06-23T01:04:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.282+0000 7fd8ec85b700 1 -- 172.21.15.5:0/2462931577 wait complete. 2023-06-23T01:04:44.297 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T01:04:40.075004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-23T01:04:40.525837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-06-23T01:04:44.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T01:04:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.416+0000 7f37debc0700 1 Processor -- start 2023-06-23T01:04:44.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.416+0000 7f37debc0700 1 -- start start 2023-06-23T01:04:44.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37debc0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d80abf40 0x7f37d81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37debc0700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f37d80ac310 0x7f37d8155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37debc0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f37d8070140 0x7f37d80c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37debc0700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d8156020 con 0x7f37d8070140 2023-06-23T01:04:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37debc0700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d815a300 con 0x7f37d80ac310 2023-06-23T01:04:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37debc0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81596f0 con 0x7f37d80abf40 2023-06-23T01:04:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37dd15d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f37d8070140 0x7f37d80c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37dd15d700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f37d8070140 0x7f37d80c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43952/0 (socket says 172.21.15.5:43952) 2023-06-23T01:04:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37dd15d700 1 -- 172.21.15.5:0/2069720969 learned_addr learned my addr 172.21.15.5:0/2069720969 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37d7fff700 1 --2- 172.21.15.5:0/2069720969 >> v2:172.21.15.152:3300/0 conn(0x7f37d80ac310 0x7f37d8155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.417+0000 7f37dc95c700 1 --2- 172.21.15.5:0/2069720969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d80abf40 0x7f37d81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37dd15d700 1 -- 172.21.15.5:0/2069720969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d80abf40 msgr2=0x7f37d81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37dd15d700 1 --2- 172.21.15.5:0/2069720969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d80abf40 0x7f37d81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37dd15d700 1 -- 172.21.15.5:0/2069720969 >> v2:172.21.15.152:3300/0 conn(0x7f37d80ac310 msgr2=0x7f37d8155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37dd15d700 1 --2- 172.21.15.5:0/2069720969 >> v2:172.21.15.152:3300/0 conn(0x7f37d80ac310 0x7f37d8155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37dd15d700 1 -- 172.21.15.5:0/2069720969 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d81599d0 con 0x7f37d8070140 2023-06-23T01:04:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37dd15d700 1 --2- 172.21.15.5:0/2069720969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8070140 0x7f37d80c9090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f37d80cd7e0 tx=0x7f37d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281153ec72f63c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37d77fe700 1 -- 172.21.15.5:0/2069720969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d000e070 con 0x7f37d8070140 2023-06-23T01:04:44.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37d77fe700 1 -- 172.21.15.5:0/2069720969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d000b970 con 0x7f37d8070140 2023-06-23T01:04:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37d77fe700 1 -- 172.21.15.5:0/2069720969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d00092e0 con 0x7f37d8070140 2023-06-23T01:04:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37debc0700 1 -- 172.21.15.5:0/2069720969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8070140 msgr2=0x7f37d80c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37debc0700 1 --2- 172.21.15.5:0/2069720969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8070140 0x7f37d80c9090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f37d80cd7e0 tx=0x7f37d0009840 comp rx=0 tx=0).stop 2023-06-23T01:04:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37debc0700 1 -- 172.21.15.5:0/2069720969 shutdown_connections 2023-06-23T01:04:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37debc0700 1 --2- 172.21.15.5:0/2069720969 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d80abf40 0x7f37d81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37debc0700 1 --2- 172.21.15.5:0/2069720969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d8070140 0x7f37d80c9090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37debc0700 1 --2- 172.21.15.5:0/2069720969 >> v2:172.21.15.152:3300/0 conn(0x7f37d80ac310 0x7f37d8155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37debc0700 1 -- 172.21.15.5:0/2069720969 >> 172.21.15.5:0/2069720969 conn(0x7f37d806e830 msgr2=0x7f37d806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:44.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.418+0000 7f37debc0700 1 -- 172.21.15.5:0/2069720969 shutdown_connections 2023-06-23T01:04:44.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.419+0000 7f37debc0700 1 -- 172.21.15.5:0/2069720969 wait complete. 2023-06-23T01:04:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.420+0000 7f37debc0700 1 Processor -- start 2023-06-23T01:04:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.420+0000 7f37debc0700 1 -- start start 2023-06-23T01:04:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.420+0000 7f37debc0700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d8070140 0x7f37d8150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.420+0000 7f37debc0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d80abf40 0x7f37d8157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.420+0000 7f37debc0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d80ac310 0x7f37d81522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.420+0000 7f37debc0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d814e960 con 0x7f37d80ac310 2023-06-23T01:04:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.420+0000 7f37debc0700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81509d0 con 0x7f37d80abf40 2023-06-23T01:04:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37debc0700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81561f0 con 0x7f37d8070140 2023-06-23T01:04:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d80ac310 0x7f37d81522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d80ac310 0x7f37d81522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43968/0 (socket says 172.21.15.5:43968) 2023-06-23T01:04:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 -- 172.21.15.5:0/2413451086 learned_addr learned my addr 172.21.15.5:0/2413451086 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37d7fff700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d80abf40 0x7f37d8157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dc95c700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d8070140 0x7f37d8150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 -- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d8070140 msgr2=0x7f37d8150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d8070140 0x7f37d8150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 -- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d80abf40 msgr2=0x7f37d8157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d80abf40 0x7f37d8157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 -- 172.21.15.5:0/2413451086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d0009ae0 con 0x7f37d80ac310 2023-06-23T01:04:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37dd15d700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d80ac310 0x7f37d81522e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009880 tx=0x7f37d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37d4ff9700 1 -- 172.21.15.5:0/2413451086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0008b10 con 0x7f37d80ac310 2023-06-23T01:04:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37d4ff9700 1 -- 172.21.15.5:0/2413451086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d0008c90 con 0x7f37d80ac310 2023-06-23T01:04:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37d4ff9700 1 -- 172.21.15.5:0/2413451086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d001c660 con 0x7f37d80ac310 2023-06-23T01:04:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d8168af0 con 0x7f37d80ac310 2023-06-23T01:04:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.421+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d81690b0 con 0x7f37d80ac310 2023-06-23T01:04:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.422+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d804c1b0 con 0x7f37d80ac310 2023-06-23T01:04:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.423+0000 7f37d4ff9700 1 -- 172.21.15.5:0/2413451086 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f37d0003040 con 0x7f37d80ac310 2023-06-23T01:04:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.423+0000 7f37d4ff9700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f37c4066e90 0x7f37c4069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.423+0000 7f37d4ff9700 1 -- 172.21.15.5:0/2413451086 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f37d00859a0 con 0x7f37d80ac310 2023-06-23T01:04:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.423+0000 7f37dc95c700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f37c4066e90 0x7f37c4069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.423+0000 7f37dc95c700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f37c4066e90 0x7f37c4069340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009db0 tx=0x7f37c80091a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.427+0000 7f37d4ff9700 1 -- 172.21.15.5:0/2413451086 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f37d0052ec0 con 0x7f37d80ac310 2023-06-23T01:04:44.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.623+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37d8156330 con 0x7f37d80ac310 2023-06-23T01:04:44.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.623+0000 7f37d4ff9700 1 -- 172.21.15.5:0/2413451086 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37d00564e0 con 0x7f37d80ac310 2023-06-23T01:04:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-06-23T01:04:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.625+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f37c4066e90 msgr2=0x7f37c4069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f37c4066e90 0x7f37c4069340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009db0 tx=0x7f37c80091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d80ac310 msgr2=0x7f37d81522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d80ac310 0x7f37d81522e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009880 tx=0x7f37d0018000 comp rx=0 tx=0).stop 2023-06-23T01:04:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 shutdown_connections 2023-06-23T01:04:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f37c4066e90 0x7f37c4069340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37d80ac310 0x7f37d81522e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f37d8070140 0x7f37d8150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 --2- 172.21.15.5:0/2413451086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d80abf40 0x7f37d8157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 >> 172.21.15.5:0/2413451086 conn(0x7f37d806e830 msgr2=0x7f37d8154ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 shutdown_connections 2023-06-23T01:04:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.626+0000 7f37debc0700 1 -- 172.21.15.5:0/2413451086 wait complete. 2023-06-23T01:04:44.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-23T01:04:44.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.759+0000 7fde1a6e4700 1 Processor -- start 2023-06-23T01:04:44.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.759+0000 7fde1a6e4700 1 -- start start 2023-06-23T01:04:44.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.759+0000 7fde1a6e4700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde140cede0 0x7fde1415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.759+0000 7fde1a6e4700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fde140b2920 0x7fde1406ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.759+0000 7fde1a6e4700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1415cf70 con 0x7fde140b2550 2023-06-23T01:04:44.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.759+0000 7fde1a6e4700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde140cb470 con 0x7fde140cede0 2023-06-23T01:04:44.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.759+0000 7fde1a6e4700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde140cb750 con 0x7fde140b2920 2023-06-23T01:04:44.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde13fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fde140b2550 0x7fde140ca8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45878/0 (socket says 172.21.15.5:45878) 2023-06-23T01:04:44.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde137fe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde140cede0 0x7fde1415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde13fff700 1 -- 172.21.15.5:0/2749587262 learned_addr learned my addr 172.21.15.5:0/2749587262 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:44.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde18c81700 1 --2- 172.21.15.5:0/2749587262 >> v2:172.21.15.152:3300/0 conn(0x7fde140b2920 0x7fde1406ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde137fe700 1 -- 172.21.15.5:0/2749587262 >> v2:172.21.15.152:3300/0 conn(0x7fde140b2920 msgr2=0x7fde1406ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:44.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde137fe700 1 --2- 172.21.15.5:0/2749587262 >> v2:172.21.15.152:3300/0 conn(0x7fde140b2920 0x7fde1406ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde137fe700 1 -- 172.21.15.5:0/2749587262 >> v1:172.21.15.5:6789/0 conn(0x7fde140b2550 legacy=0x7fde140ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde12ffd700 1 -- 172.21.15.5:0/2749587262 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676088183 0 0) 0x7fde1415cf70 con 0x7fde140b2550 2023-06-23T01:04:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde137fe700 1 -- 172.21.15.5:0/2749587262 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1415d1c0 con 0x7fde140cede0 2023-06-23T01:04:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde137fe700 1 --2- 172.21.15.5:0/2749587262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140cede0 0x7fde1415d9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fde1406bab0 tx=0x7fde080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55b0d2ea41644c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde12ffd700 1 -- 172.21.15.5:0/2749587262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08011630 con 0x7fde140cede0 2023-06-23T01:04:44.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde12ffd700 1 -- 172.21.15.5:0/2749587262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde08011790 con 0x7fde140cede0 2023-06-23T01:04:44.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.760+0000 7fde12ffd700 1 -- 172.21.15.5:0/2749587262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08011b30 con 0x7fde140cede0 2023-06-23T01:04:44.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.761+0000 7fde1a6e4700 1 -- 172.21.15.5:0/2749587262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140cede0 msgr2=0x7fde1415d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.761+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/2749587262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140cede0 0x7fde1415d9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fde1406bab0 tx=0x7fde080091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.761+0000 7fde1a6e4700 1 -- 172.21.15.5:0/2749587262 shutdown_connections 2023-06-23T01:04:44.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.761+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/2749587262 >> v2:172.21.15.152:3300/0 conn(0x7fde140b2920 0x7fde1406ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.761+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/2749587262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140cede0 0x7fde1415d9c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.761+0000 7fde1a6e4700 1 -- 172.21.15.5:0/2749587262 >> 172.21.15.5:0/2749587262 conn(0x7fde140d02a0 msgr2=0x7fde140d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:44.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.761+0000 7fde1a6e4700 1 -- 172.21.15.5:0/2749587262 shutdown_connections 2023-06-23T01:04:44.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.761+0000 7fde1a6e4700 1 -- 172.21.15.5:0/2749587262 wait complete. 2023-06-23T01:04:44.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.762+0000 7fde1a6e4700 1 Processor -- start 2023-06-23T01:04:44.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.762+0000 7fde1a6e4700 1 -- start start 2023-06-23T01:04:44.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde1a6e4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140b2550 0x7fde14169de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde1a6e4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde140b2920 0x7fde1415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde1a6e4700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde140cede0 0x7fde141565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde1a6e4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1415fb60 con 0x7fde140b2550 2023-06-23T01:04:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde1a6e4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1415ef50 con 0x7fde140b2920 2023-06-23T01:04:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde1a6e4700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1415f230 con 0x7fde140cede0 2023-06-23T01:04:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde13fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140b2550 0x7fde14169de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde13fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140b2550 0x7fde14169de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33866/0 (socket says 172.21.15.5:33866) 2023-06-23T01:04:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde13fff700 1 -- 172.21.15.5:0/3163753042 learned_addr learned my addr 172.21.15.5:0/3163753042 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde18c81700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde140cede0 0x7fde141565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde137fe700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde140b2920 0x7fde1415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde13fff700 1 -- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde140cede0 msgr2=0x7fde141565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde13fff700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde140cede0 0x7fde141565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde13fff700 1 -- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde140b2920 msgr2=0x7fde1415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde13fff700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde140b2920 0x7fde1415e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde13fff700 1 -- 172.21.15.5:0/3163753042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08009420 con 0x7fde140b2550 2023-06-23T01:04:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.763+0000 7fde137fe700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde140b2920 0x7fde1415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.764+0000 7fde13fff700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140b2550 0x7fde14169de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde0400c5b0 tx=0x7fde04001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.764+0000 7fddfbfff700 1 -- 172.21.15.5:0/3163753042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04005b30 con 0x7fde140b2550 2023-06-23T01:04:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.764+0000 7fddfbfff700 1 -- 172.21.15.5:0/3163753042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04005c90 con 0x7fde140b2550 2023-06-23T01:04:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.764+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1415f510 con 0x7fde140b2550 2023-06-23T01:04:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.764+0000 7fddfbfff700 1 -- 172.21.15.5:0/3163753042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04007300 con 0x7fde140b2550 2023-06-23T01:04:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.764+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde141694a0 con 0x7fde140b2550 2023-06-23T01:04:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.765+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1404c1b0 con 0x7fde140b2550 2023-06-23T01:04:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.765+0000 7fddfbfff700 1 -- 172.21.15.5:0/3163753042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fde0401b3b0 con 0x7fde140b2550 2023-06-23T01:04:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.766+0000 7fddfbfff700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fde00066e90 0x7fde00069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.766+0000 7fddfbfff700 1 -- 172.21.15.5:0/3163753042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fde04083790 con 0x7fde140b2550 2023-06-23T01:04:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.766+0000 7fde137fe700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fde00066e90 0x7fde00069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.766+0000 7fde137fe700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fde00066e90 0x7fde00069340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde140cf1e0 tx=0x7fde08017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.770+0000 7fddfbfff700 1 -- 172.21.15.5:0/3163753042 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fde04050cb0 con 0x7fde140b2550 2023-06-23T01:04:44.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.963+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fde140ce020 con 0x7fde140b2550 2023-06-23T01:04:44.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.963+0000 7fddfbfff700 1 -- 172.21.15.5:0/3163753042 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fde040542d0 con 0x7fde140b2550 2023-06-23T01:04:44.967 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 8 2023-06-23T01:04:44.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fde00066e90 msgr2=0x7fde00069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fde00066e90 0x7fde00069340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde140cf1e0 tx=0x7fde08017000 comp rx=0 tx=0).stop 2023-06-23T01:04:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140b2550 msgr2=0x7fde14169de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140b2550 0x7fde14169de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde0400c5b0 tx=0x7fde04001070 comp rx=0 tx=0).stop 2023-06-23T01:04:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 shutdown_connections 2023-06-23T01:04:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fde00066e90 0x7fde00069340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fde140cede0 0x7fde141565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde140b2550 0x7fde14169de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 --2- 172.21.15.5:0/3163753042 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde140b2920 0x7fde1415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 >> 172.21.15.5:0/3163753042 conn(0x7fde140d02a0 msgr2=0x7fde1406ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 shutdown_connections 2023-06-23T01:04:44.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:44.965+0000 7fde1a6e4700 1 -- 172.21.15.5:0/3163753042 wait complete. 2023-06-23T01:04:44.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-23T01:04:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.096+0000 7f8c00621700 1 Processor -- start 2023-06-23T01:04:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.096+0000 7f8c00621700 1 -- start start 2023-06-23T01:04:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8c00621700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8bf80ab830 0x7f8bf806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8c00621700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8bf81596a0 0x7f8bf815d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8c00621700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf806cd70 con 0x7f8bf81592d0 2023-06-23T01:04:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8c00621700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80ac8d0 con 0x7f8bf81596a0 2023-06-23T01:04:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8c00621700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf815de50 con 0x7f8bf80ab830 2023-06-23T01:04:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8bfebbe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8bf81596a0 0x7f8bf815d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8bfebbe700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8bf81596a0 0x7f8bf815d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33872/0 (socket says 172.21.15.5:33872) 2023-06-23T01:04:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8bfebbe700 1 -- 172.21.15.5:0/3159700478 learned_addr learned my addr 172.21.15.5:0/3159700478 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8bfdbbc700 1 --2- 172.21.15.5:0/3159700478 >> v2:172.21.15.152:3300/0 conn(0x7f8bf80ab830 0x7f8bf806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8bfebbe700 1 -- 172.21.15.5:0/3159700478 >> v2:172.21.15.152:3300/0 conn(0x7f8bf80ab830 msgr2=0x7f8bf806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8bfebbe700 1 --2- 172.21.15.5:0/3159700478 >> v2:172.21.15.152:3300/0 conn(0x7f8bf80ab830 0x7f8bf806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8bfebbe700 1 -- 172.21.15.5:0/3159700478 >> v1:172.21.15.152:6789/0 conn(0x7f8bf81592d0 legacy=0x7f8bf80af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.097+0000 7f8bfebbe700 1 -- 172.21.15.5:0/3159700478 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80af810 con 0x7f8bf81596a0 2023-06-23T01:04:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8bfebbe700 1 --2- 172.21.15.5:0/3159700478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815d050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8064580 tx=0x7f8be8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91078f4a4a84eb58 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8bfd3bb700 1 -- 172.21.15.5:0/3159700478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800e070 con 0x7f8bf81596a0 2023-06-23T01:04:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8bfd3bb700 1 -- 172.21.15.5:0/3159700478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be800ba20 con 0x7f8bf81596a0 2023-06-23T01:04:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8bfd3bb700 1 -- 172.21.15.5:0/3159700478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80092e0 con 0x7f8bf81596a0 2023-06-23T01:04:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8c00621700 1 -- 172.21.15.5:0/3159700478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 msgr2=0x7f8bf815d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8c00621700 1 --2- 172.21.15.5:0/3159700478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815d050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8064580 tx=0x7f8be8009840 comp rx=0 tx=0).stop 2023-06-23T01:04:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8c00621700 1 -- 172.21.15.5:0/3159700478 shutdown_connections 2023-06-23T01:04:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8c00621700 1 --2- 172.21.15.5:0/3159700478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815d050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8c00621700 1 --2- 172.21.15.5:0/3159700478 >> v2:172.21.15.152:3300/0 conn(0x7f8bf80ab830 0x7f8bf806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8c00621700 1 -- 172.21.15.5:0/3159700478 >> 172.21.15.5:0/3159700478 conn(0x7f8bf806f190 msgr2=0x7f8bf806c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8c00621700 1 -- 172.21.15.5:0/3159700478 shutdown_connections 2023-06-23T01:04:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.098+0000 7f8c00621700 1 -- 172.21.15.5:0/3159700478 wait complete. 2023-06-23T01:04:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.099+0000 7f8c00621700 1 Processor -- start 2023-06-23T01:04:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.099+0000 7f8c00621700 1 -- start start 2023-06-23T01:04:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8c00621700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf80ab830 0x7f8bf815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8c00621700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8bf81592d0 0x7f8bf8142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8c00621700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8c00621700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf815a7e0 con 0x7f8bf81596a0 2023-06-23T01:04:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8c00621700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8159bd0 con 0x7f8bf80ab830 2023-06-23T01:04:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8c00621700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8159eb0 con 0x7f8bf81592d0 2023-06-23T01:04:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8bfebbe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8bfebbe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33884/0 (socket says 172.21.15.5:33884) 2023-06-23T01:04:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8bfebbe700 1 -- 172.21.15.5:0/3643354647 learned_addr learned my addr 172.21.15.5:0/3643354647 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8bfdbbc700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8bf81592d0 0x7f8bf8142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.100+0000 7f8bfe3bd700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf80ab830 0x7f8bf815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bfebbe700 1 -- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8bf81592d0 msgr2=0x7f8bf8142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bfebbe700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8bf81592d0 0x7f8bf8142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bfebbe700 1 -- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf80ab830 msgr2=0x7f8bf815c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bfebbe700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf80ab830 0x7f8bf815c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bfebbe700 1 -- 172.21.15.5:0/3643354647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be8009ae0 con 0x7f8bf81596a0 2023-06-23T01:04:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bfebbe700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815ac70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009880 tx=0x7f8be8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bf67fc700 1 -- 172.21.15.5:0/3643354647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8008a80 con 0x7f8bf81596a0 2023-06-23T01:04:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bf67fc700 1 -- 172.21.15.5:0/3643354647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be8008be0 con 0x7f8bf81596a0 2023-06-23T01:04:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf815a190 con 0x7f8bf81596a0 2023-06-23T01:04:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8bf67fc700 1 -- 172.21.15.5:0/3643354647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8025460 con 0x7f8bf81596a0 2023-06-23T01:04:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.101+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf815c9a0 con 0x7f8bf81596a0 2023-06-23T01:04:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.102+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf804c1b0 con 0x7f8bf81596a0 2023-06-23T01:04:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.102+0000 7f8bf67fc700 1 -- 172.21.15.5:0/3643354647 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f8be8003040 con 0x7f8bf81596a0 2023-06-23T01:04:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.102+0000 7f8bf67fc700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f8be4066e90 0x7f8be4069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.103+0000 7f8bf67fc700 1 -- 172.21.15.5:0/3643354647 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8be8085d10 con 0x7f8bf81596a0 2023-06-23T01:04:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.103+0000 7f8bfe3bd700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f8be4066e90 0x7f8be4069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.103+0000 7f8bfe3bd700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f8be4066e90 0x7f8be4069340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0003a80 tx=0x7f8bf0007ec0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.107+0000 7f8bf67fc700 1 -- 172.21.15.5:0/3643354647 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f8be80532e0 con 0x7f8bf81596a0 2023-06-23T01:04:45.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.302+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8bf80abcc0 con 0x7f8bf81596a0 2023-06-23T01:04:45.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.303+0000 7f8bf67fc700 1 -- 172.21.15.5:0/3643354647 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8be8020090 con 0x7f8bf81596a0 2023-06-23T01:04:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:04:45.307 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2023-06-23T01:04:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-06-23T01:04:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2023-06-23T01:04:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2023-06-23T01:04:45.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-06-23T01:04:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-06-23T01:04:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 2, 2023-06-23T01:04:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": "rbd" 2023-06-23T01:04:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-06-23T01:04:45.308 INFO:teuthology.orchestra.run.smithi005.stdout:] 2023-06-23T01:04:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f8be4066e90 msgr2=0x7f8be4069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f8be4066e90 0x7f8be4069340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0003a80 tx=0x7f8bf0007ec0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 msgr2=0x7f8bf815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815ac70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009880 tx=0x7f8be8018000 comp rx=0 tx=0).stop 2023-06-23T01:04:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 shutdown_connections 2023-06-23T01:04:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f8be4066e90 0x7f8be4069340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bf81596a0 0x7f8bf815ac70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf80ab830 0x7f8bf815c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 --2- 172.21.15.5:0/3643354647 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8bf81592d0 0x7f8bf8142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 >> 172.21.15.5:0/3643354647 conn(0x7f8bf806f190 msgr2=0x7f8bf806a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 shutdown_connections 2023-06-23T01:04:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.305+0000 7f8c00621700 1 -- 172.21.15.5:0/3643354647 wait complete. 2023-06-23T01:04:45.320 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-23T01:04:45.320 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-23T01:04:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.440+0000 7f444e270700 1 Processor -- start 2023-06-23T01:04:45.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.440+0000 7f444e270700 1 -- start start 2023-06-23T01:04:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f444e270700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f444815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f444e270700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44480afce0 con 0x7f4448153780 2023-06-23T01:04:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f444e270700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444815dbb0 con 0x7f44480ccc70 2023-06-23T01:04:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f444e270700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444815cfa0 con 0x7f4448153b50 2023-06-23T01:04:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f4447fff700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4448153780 0x7f44480cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45890/0 (socket says 172.21.15.5:45890) 2023-06-23T01:04:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f4447fff700 1 -- 172.21.15.5:0/1714999732 learned_addr learned my addr 172.21.15.5:0/1714999732 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f44477fe700 1 --2- 172.21.15.5:0/1714999732 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f444815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f4446ffd700 1 -- 172.21.15.5:0/1714999732 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723063035 0 0) 0x7f44480afce0 con 0x7f4448153780 2023-06-23T01:04:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f4446ffd700 1 -- 172.21.15.5:0/1714999732 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4430003570 con 0x7f4448153780 2023-06-23T01:04:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f4446ffd700 1 -- 172.21.15.5:0/1714999732 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778589067 0 0) 0x7f444815dbb0 con 0x7f44480ccc70 2023-06-23T01:04:45.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f4446ffd700 1 -- 172.21.15.5:0/1714999732 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44480afce0 con 0x7f44480ccc70 2023-06-23T01:04:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f44477fe700 1 -- 172.21.15.5:0/1714999732 >> v1:172.21.15.152:6789/0 conn(0x7f44480ccc70 legacy=0x7f44480aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f4446ffd700 1 -- 172.21.15.5:0/1714999732 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 623677281 0 0) 0x7f4430003570 con 0x7f4448153780 2023-06-23T01:04:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f44477fe700 1 -- 172.21.15.5:0/1714999732 >> v1:172.21.15.5:6789/0 conn(0x7f4448153780 legacy=0x7f44480cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f44477fe700 1 -- 172.21.15.5:0/1714999732 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444815d280 con 0x7f4448153b50 2023-06-23T01:04:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.441+0000 7f44477fe700 1 --2- 172.21.15.5:0/1714999732 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f444815a4d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f443c009910 tx=0x7f443c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68debd0a4b2076b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f4446ffd700 1 -- 172.21.15.5:0/1714999732 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438001470 con 0x7f4448153b50 2023-06-23T01:04:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f4446ffd700 1 -- 172.21.15.5:0/1714999732 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c011420 con 0x7f4448153b50 2023-06-23T01:04:45.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f4446ffd700 1 -- 172.21.15.5:0/1714999732 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c0117a0 con 0x7f4448153b50 2023-06-23T01:04:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f444e270700 1 -- 172.21.15.5:0/1714999732 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 msgr2=0x7f444815a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f444e270700 1 --2- 172.21.15.5:0/1714999732 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f444815a4d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f443c009910 tx=0x7f443c0091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f444e270700 1 -- 172.21.15.5:0/1714999732 shutdown_connections 2023-06-23T01:04:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f444e270700 1 --2- 172.21.15.5:0/1714999732 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f444815a4d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f444e270700 1 -- 172.21.15.5:0/1714999732 >> 172.21.15.5:0/1714999732 conn(0x7f4448070be0 msgr2=0x7f444806d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f444e270700 1 -- 172.21.15.5:0/1714999732 shutdown_connections 2023-06-23T01:04:45.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.442+0000 7f444e270700 1 -- 172.21.15.5:0/1714999732 wait complete. 2023-06-23T01:04:45.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.443+0000 7f444e270700 1 Processor -- start 2023-06-23T01:04:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.443+0000 7f444e270700 1 -- start start 2023-06-23T01:04:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f444e270700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480ccc70 0x7f4448151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f444e270700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4448153780 0x7f4448152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f444e270700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f4448152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f444e270700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4448151880 con 0x7f44480ccc70 2023-06-23T01:04:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f444e270700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44480c8330 con 0x7f4448153780 2023-06-23T01:04:45.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f444e270700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44480c8610 con 0x7f4448153b50 2023-06-23T01:04:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f4447fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480ccc70 0x7f4448151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f4447fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480ccc70 0x7f4448151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33886/0 (socket says 172.21.15.5:33886) 2023-06-23T01:04:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f4447fff700 1 -- 172.21.15.5:0/3855677341 learned_addr learned my addr 172.21.15.5:0/3855677341 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f444c80d700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f4448152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.444+0000 7f44477fe700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4448153780 0x7f4448152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f4447fff700 1 -- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 msgr2=0x7f4448152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f4447fff700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f4448152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f4447fff700 1 -- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4448153780 msgr2=0x7f4448152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:45.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f4447fff700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4448153780 0x7f4448152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:45.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f4447fff700 1 -- 172.21.15.5:0/3855677341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009420 con 0x7f44480ccc70 2023-06-23T01:04:45.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f444c80d700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f4448152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:04:45.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f4447fff700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480ccc70 0x7f4448151340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f443800cb60 tx=0x7f4438001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:45.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f442ffff700 1 -- 172.21.15.5:0/3855677341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438007e90 con 0x7f44480ccc70 2023-06-23T01:04:45.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f442ffff700 1 -- 172.21.15.5:0/3855677341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4438007360 con 0x7f44480ccc70 2023-06-23T01:04:45.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44480b3d60 con 0x7f44480ccc70 2023-06-23T01:04:45.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f442ffff700 1 -- 172.21.15.5:0/3855677341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438004830 con 0x7f44480ccc70 2023-06-23T01:04:45.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.445+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44480b4320 con 0x7f44480ccc70 2023-06-23T01:04:45.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.446+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444804c1b0 con 0x7f44480ccc70 2023-06-23T01:04:45.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.446+0000 7f442ffff700 1 -- 172.21.15.5:0/3855677341 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f4438004ba0 con 0x7f44480ccc70 2023-06-23T01:04:45.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.447+0000 7f442ffff700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f4434066e90 0x7f4434069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:45.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.447+0000 7f442ffff700 1 -- 172.21.15.5:0/3855677341 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f44380836d0 con 0x7f44480ccc70 2023-06-23T01:04:45.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.447+0000 7f44477fe700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f4434066e90 0x7f4434069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:45.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.447+0000 7f44477fe700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f4434066e90 0x7f4434069340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4448153f80 tx=0x7f443c016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:45.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.451+0000 7f442ffff700 1 -- 172.21.15.5:0/3855677341 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f4438050c70 con 0x7f44480ccc70 2023-06-23T01:04:45.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:45.645+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f444815d560 con 0x7f44480ccc70 2023-06-23T01:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.612+0000 7f442ffff700 1 -- 172.21.15.5:0/3855677341 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4438018020 con 0x7f44480ccc70 2023-06-23T01:04:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f4434066e90 msgr2=0x7f4434069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f4434066e90 0x7f4434069340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4448153f80 tx=0x7f443c016000 comp rx=0 tx=0).stop 2023-06-23T01:04:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480ccc70 msgr2=0x7f4448151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480ccc70 0x7f4448151340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f443800cb60 tx=0x7f4438001ad0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 shutdown_connections 2023-06-23T01:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f4434066e90 0x7f4434069340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4448153b50 0x7f4448152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44480ccc70 0x7f4448151340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 --2- 172.21.15.5:0/3855677341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4448153780 0x7f4448152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.615+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 >> 172.21.15.5:0/3855677341 conn(0x7f4448070be0 msgr2=0x7f44480c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:46.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.616+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 shutdown_connections 2023-06-23T01:04:46.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.616+0000 7f444e270700 1 -- 172.21.15.5:0/3855677341 wait complete. 2023-06-23T01:04:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2023-06-23T01:04:46.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-23T01:04:46.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.766+0000 7ff559214700 1 Processor -- start 2023-06-23T01:04:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.766+0000 7ff559214700 1 -- start start 2023-06-23T01:04:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.767+0000 7ff559214700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff5540cd9c0 0x7ff5540cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.767+0000 7ff559214700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55415ada0 con 0x7ff554156f60 2023-06-23T01:04:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.767+0000 7ff559214700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55415b920 con 0x7ff554068770 2023-06-23T01:04:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.767+0000 7ff559214700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55415bdb0 con 0x7ff5540cd9c0 2023-06-23T01:04:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.767+0000 7ff552d9d700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff554156f60 0x7ff554068060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45906/0 (socket says 172.21.15.5:45906) 2023-06-23T01:04:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.767+0000 7ff552d9d700 1 -- 172.21.15.5:0/2772637173 learned_addr learned my addr 172.21.15.5:0/2772637173 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.767+0000 7ff55359e700 1 --2- 172.21.15.5:0/2772637173 >> v2:172.21.15.152:3300/0 conn(0x7ff5540cd9c0 0x7ff5540cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798889834 0 0) 0x7ff55415ada0 con 0x7ff554156f60 2023-06-23T01:04:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff540003570 con 0x7ff554156f60 2023-06-23T01:04:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3753125209 0 0) 0x7ff540003570 con 0x7ff554156f60 2023-06-23T01:04:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 >> v2:172.21.15.152:3300/0 conn(0x7ff5540cd9c0 msgr2=0x7ff5540cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 --2- 172.21.15.5:0/2772637173 >> v2:172.21.15.152:3300/0 conn(0x7ff5540cd9c0 0x7ff5540cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 >> v1:172.21.15.152:6789/0 conn(0x7ff554068770 legacy=0x7ff5540ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55415aff0 con 0x7ff554156f60 2023-06-23T01:04:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1394223029 0 0) 0x7ff54c001470 con 0x7ff554156f60 2023-06-23T01:04:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 >> v1:172.21.15.5:6789/0 conn(0x7ff554156f60 legacy=0x7ff554068060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff540003a60 0x7ff540003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff540004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5400051c0 0x7ff54000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54000c050 con 0x7ff540004810 2023-06-23T01:04:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54000c190 con 0x7ff5400051c0 2023-06-23T01:04:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54000c350 con 0x7ff540003a60 2023-06-23T01:04:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55259c700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff540004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff552d9d700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff540003a60 0x7ff540003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55359e700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5400051c0 0x7ff54000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55259c700 1 -- 172.21.15.5:0/2772637173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff540003a60 msgr2=0x7ff540003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55259c700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff540003a60 0x7ff540003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55259c700 1 -- 172.21.15.5:0/2772637173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5400051c0 msgr2=0x7ff54000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55259c700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff5400051c0 0x7ff54000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55359e700 1 -- 172.21.15.5:0/2772637173 reap_dead start 2023-06-23T01:04:46.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55259c700 1 -- 172.21.15.5:0/2772637173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540003570 con 0x7ff540004810 2023-06-23T01:04:46.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.768+0000 7ff55259c700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff540004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001970 tx=0x7ff53c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba848cc5be984e3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:46.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00c5d0 con 0x7ff540004810 2023-06-23T01:04:46.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c00c730 con 0x7ff540004810 2023-06-23T01:04:46.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff551d9b700 1 -- 172.21.15.5:0/2772637173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00cad0 con 0x7ff540004810 2023-06-23T01:04:46.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff559214700 1 -- 172.21.15.5:0/2772637173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 msgr2=0x7ff540004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:46.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff559214700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff540004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001970 tx=0x7ff53c007d10 comp rx=0 tx=0).stop 2023-06-23T01:04:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff559214700 1 -- 172.21.15.5:0/2772637173 shutdown_connections 2023-06-23T01:04:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff559214700 1 --2- 172.21.15.5:0/2772637173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff540004c80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff559214700 1 -- 172.21.15.5:0/2772637173 >> 172.21.15.5:0/2772637173 conn(0x7ff55406c000 msgr2=0x7ff55406c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff559214700 1 -- 172.21.15.5:0/2772637173 shutdown_connections 2023-06-23T01:04:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.769+0000 7ff559214700 1 -- 172.21.15.5:0/2772637173 wait complete. 2023-06-23T01:04:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.770+0000 7ff559214700 1 Processor -- start 2023-06-23T01:04:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.770+0000 7ff559214700 1 -- start start 2023-06-23T01:04:46.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff559214700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff554169530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:46.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff559214700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff554146a00 0x7ff554153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:46.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff559214700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff554154390 0x7ff554155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:46.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff559214700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554159350 con 0x7ff540004810 2023-06-23T01:04:46.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff559214700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554158460 con 0x7ff554146a00 2023-06-23T01:04:46.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff559214700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff554158740 con 0x7ff554154390 2023-06-23T01:04:46.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff552d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff554169530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:46.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff552d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff554169530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33916/0 (socket says 172.21.15.5:33916) 2023-06-23T01:04:46.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff552d9d700 1 -- 172.21.15.5:0/2458991395 learned_addr learned my addr 172.21.15.5:0/2458991395 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:46.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff55259c700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff554146a00 0x7ff554153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:46.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.771+0000 7ff55359e700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff554154390 0x7ff554155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:46.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff552d9d700 1 -- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff554154390 msgr2=0x7ff554155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:46.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff552d9d700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff554154390 0x7ff554155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff552d9d700 1 -- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff554146a00 msgr2=0x7ff554153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:46.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff552d9d700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff554146a00 0x7ff554153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:46.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff552d9d700 1 -- 172.21.15.5:0/2458991395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554158a20 con 0x7ff540004810 2023-06-23T01:04:46.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff552d9d700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff554169530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff54c007390 tx=0x7ff54c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff54affd700 1 -- 172.21.15.5:0/2458991395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c018030 con 0x7ff540004810 2023-06-23T01:04:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff54affd700 1 -- 172.21.15.5:0/2458991395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c007770 con 0x7ff540004810 2023-06-23T01:04:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff54affd700 1 -- 172.21.15.5:0/2458991395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00fa40 con 0x7ff540004810 2023-06-23T01:04:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff554158d00 con 0x7ff540004810 2023-06-23T01:04:46.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.772+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff554169a70 con 0x7ff540004810 2023-06-23T01:04:46.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.773+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff554154760 con 0x7ff540004810 2023-06-23T01:04:46.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.773+0000 7ff54affd700 1 -- 172.21.15.5:0/2458991395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7ff54c01b070 con 0x7ff540004810 2023-06-23T01:04:46.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.774+0000 7ff54affd700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7ff534066e90 0x7ff534069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:46.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.774+0000 7ff54affd700 1 -- 172.21.15.5:0/2458991395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5576+0+0 (secure 0 0 0) 0x7ff54c019070 con 0x7ff540004810 2023-06-23T01:04:46.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.774+0000 7ff55259c700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7ff534066e90 0x7ff534069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:46.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.774+0000 7ff55259c700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7ff534066e90 0x7ff534069340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff554156980 tx=0x7ff53c015000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:46.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.778+0000 7ff54affd700 1 -- 172.21.15.5:0/2458991395 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7ff54c0515f0 con 0x7ff540004810 2023-06-23T01:04:46.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:46.972+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff5541591a0 con 0x7ff540004810 2023-06-23T01:04:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.619+0000 7ff54affd700 1 -- 172.21.15.5:0/2458991395 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff54c00fdf0 con 0x7ff540004810 2023-06-23T01:04:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.626+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7ff534066e90 msgr2=0x7ff534069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.626+0000 7ff559214700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7ff534066e90 0x7ff534069340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff554156980 tx=0x7ff53c015000 comp rx=0 tx=0).stop 2023-06-23T01:04:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.626+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 msgr2=0x7ff554169530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.626+0000 7ff559214700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff554169530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff54c007390 tx=0x7ff54c001e70 comp rx=0 tx=0).stop 2023-06-23T01:04:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.628+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 shutdown_connections 2023-06-23T01:04:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.628+0000 7ff559214700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7ff534066e90 0x7ff534069340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.628+0000 7ff559214700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff554154390 0x7ff554155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.628+0000 7ff559214700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff540004810 0x7ff554169530 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.628+0000 7ff559214700 1 --2- 172.21.15.5:0/2458991395 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff554146a00 0x7ff554153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.628+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 >> 172.21.15.5:0/2458991395 conn(0x7ff55406c000 msgr2=0x7ff5540b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:47.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.633+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 shutdown_connections 2023-06-23T01:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.634+0000 7ff559214700 1 -- 172.21.15.5:0/2458991395 wait complete. 2023-06-23T01:04:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2023-06-23T01:04:47.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-23T01:04:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.775+0000 7f1e7e825700 1 Processor -- start 2023-06-23T01:04:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.775+0000 7f1e7e825700 1 -- start start 2023-06-23T01:04:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e7e825700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1e780b2550 0x7f1e7815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e7e825700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1e780b2920 0x7f1e7806ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e7e825700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7815cf70 con 0x7f1e780cede0 2023-06-23T01:04:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e7e825700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e780cb470 con 0x7f1e780b2550 2023-06-23T01:04:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e7e825700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e780cb750 con 0x7f1e780b2920 2023-06-23T01:04:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e77fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1e780b2550 0x7f1e7815d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e77fff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1e780b2550 0x7f1e7815d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33932/0 (socket says 172.21.15.5:33932) 2023-06-23T01:04:47.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e77fff700 1 -- 172.21.15.5:0/3571630914 learned_addr learned my addr 172.21.15.5:0/3571630914 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e7cdc2700 1 --2- 172.21.15.5:0/3571630914 >> v2:172.21.15.152:3300/0 conn(0x7f1e780b2920 0x7f1e7806ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e77fff700 1 -- 172.21.15.5:0/3571630914 >> v2:172.21.15.152:3300/0 conn(0x7f1e780b2920 msgr2=0x7f1e7806ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e77fff700 1 --2- 172.21.15.5:0/3571630914 >> v2:172.21.15.152:3300/0 conn(0x7f1e780b2920 0x7f1e7806ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e77fff700 1 -- 172.21.15.5:0/3571630914 >> v1:172.21.15.152:6789/0 conn(0x7f1e780cede0 legacy=0x7f1e780ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e77fff700 1 -- 172.21.15.5:0/3571630914 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7815d1c0 con 0x7f1e780b2550 2023-06-23T01:04:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e7cdc2700 1 --2- 172.21.15.5:0/3571630914 >> v2:172.21.15.152:3300/0 conn(0x7f1e780b2920 0x7f1e7806ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:04:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e77fff700 1 --2- 172.21.15.5:0/3571630914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e7815d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e7806bab0 tx=0x7f1e68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a37f15174b3356 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.776+0000 7f1e76ffd700 1 -- 172.21.15.5:0/3571630914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800b860 con 0x7f1e780b2550 2023-06-23T01:04:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e76ffd700 1 -- 172.21.15.5:0/3571630914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6800b9c0 con 0x7f1e780b2550 2023-06-23T01:04:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e76ffd700 1 -- 172.21.15.5:0/3571630914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800bd60 con 0x7f1e780b2550 2023-06-23T01:04:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e7e825700 1 -- 172.21.15.5:0/3571630914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 msgr2=0x7f1e7815d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e7e825700 1 --2- 172.21.15.5:0/3571630914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e7815d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1e7806bab0 tx=0x7f1e68009250 comp rx=0 tx=0).stop 2023-06-23T01:04:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e7e825700 1 -- 172.21.15.5:0/3571630914 shutdown_connections 2023-06-23T01:04:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e7e825700 1 --2- 172.21.15.5:0/3571630914 >> v2:172.21.15.152:3300/0 conn(0x7f1e780b2920 0x7f1e7806ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e7e825700 1 --2- 172.21.15.5:0/3571630914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e7815d9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e7e825700 1 -- 172.21.15.5:0/3571630914 >> 172.21.15.5:0/3571630914 conn(0x7f1e780d02a0 msgr2=0x7f1e780d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e7e825700 1 -- 172.21.15.5:0/3571630914 shutdown_connections 2023-06-23T01:04:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.777+0000 7f1e7e825700 1 -- 172.21.15.5:0/3571630914 wait complete. 2023-06-23T01:04:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.778+0000 7f1e7e825700 1 Processor -- start 2023-06-23T01:04:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.778+0000 7f1e7e825700 1 -- start start 2023-06-23T01:04:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e7e825700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e780c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e7e825700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780b2920 0x7f1e780c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e7e825700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e780cede0 0x7f1e7816d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e7e825700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780c9010 con 0x7f1e780b2550 2023-06-23T01:04:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e7e825700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780c8d30 con 0x7f1e780b2920 2023-06-23T01:04:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e7e825700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e78151cc0 con 0x7f1e780cede0 2023-06-23T01:04:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e780c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e780c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33936/0 (socket says 172.21.15.5:33936) 2023-06-23T01:04:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e77fff700 1 -- 172.21.15.5:0/2626485188 learned_addr learned my addr 172.21.15.5:0/2626485188 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e7cdc2700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e780cede0 0x7f1e7816d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e777fe700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780b2920 0x7f1e780c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e77fff700 1 -- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e780cede0 msgr2=0x7f1e7816d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e77fff700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e780cede0 0x7f1e7816d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e77fff700 1 -- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780b2920 msgr2=0x7f1e780c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e77fff700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780b2920 0x7f1e780c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.779+0000 7f1e77fff700 1 -- 172.21.15.5:0/2626485188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680094f0 con 0x7f1e780b2550 2023-06-23T01:04:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.780+0000 7f1e777fe700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780b2920 0x7f1e780c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.780+0000 7f1e77fff700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e780c6010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009290 tx=0x7f1e68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.780+0000 7f1e6ffff700 1 -- 172.21.15.5:0/2626485188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68011e60 con 0x7f1e780b2550 2023-06-23T01:04:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.780+0000 7f1e6ffff700 1 -- 172.21.15.5:0/2626485188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6801f070 con 0x7f1e780b2550 2023-06-23T01:04:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.780+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e78151fa0 con 0x7f1e780b2550 2023-06-23T01:04:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.780+0000 7f1e6ffff700 1 -- 172.21.15.5:0/2626485188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6801a870 con 0x7f1e780b2550 2023-06-23T01:04:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.780+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e780c7590 con 0x7f1e780b2550 2023-06-23T01:04:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.781+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e78152440 con 0x7f1e780b2550 2023-06-23T01:04:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.781+0000 7f1e6ffff700 1 -- 172.21.15.5:0/2626485188 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f1e6801abe0 con 0x7f1e780b2550 2023-06-23T01:04:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.782+0000 7f1e6ffff700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f1e58066e90 0x7f1e58069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.782+0000 7f1e6ffff700 1 -- 172.21.15.5:0/2626485188 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f1e68021070 con 0x7f1e780b2550 2023-06-23T01:04:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.782+0000 7f1e777fe700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f1e58066e90 0x7f1e58069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.783+0000 7f1e777fe700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f1e58066e90 0x7f1e58069340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e78156900 tx=0x7f1e60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:47.786+0000 7f1e6ffff700 1 -- 172.21.15.5:0/2626485188 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f1e680536f0 con 0x7f1e780b2550 2023-06-23T01:04:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.024+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1e7804c1b0 con 0x7f1e780b2550 2023-06-23T01:04:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.030+0000 7f1e6ffff700 1 -- 172.21.15.5:0/2626485188 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1e68056d10 con 0x7f1e780b2550 2023-06-23T01:04:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.035+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f1e58066e90 msgr2=0x7f1e58069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.035+0000 7f1e7e825700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f1e58066e90 0x7f1e58069340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1e78156900 tx=0x7f1e60009040 comp rx=0 tx=0).stop 2023-06-23T01:04:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.035+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 msgr2=0x7f1e780c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.035+0000 7f1e7e825700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e780c6010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009290 tx=0x7f1e68017000 comp rx=0 tx=0).stop 2023-06-23T01:04:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.037+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 shutdown_connections 2023-06-23T01:04:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.037+0000 7f1e7e825700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f1e58066e90 0x7f1e58069340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.037+0000 7f1e7e825700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1e780cede0 0x7f1e7816d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.037+0000 7f1e7e825700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e780b2550 0x7f1e780c6010 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.037+0000 7f1e7e825700 1 --2- 172.21.15.5:0/2626485188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e780b2920 0x7f1e780c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.038+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 >> 172.21.15.5:0/2626485188 conn(0x7f1e780d02a0 msgr2=0x7f1e78156e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.038+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 shutdown_connections 2023-06-23T01:04:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.039+0000 7f1e7e825700 1 -- 172.21.15.5:0/2626485188 wait complete. 2023-06-23T01:04:48.046 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 3 and data pool 4 2023-06-23T01:04:48.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:04:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.183+0000 7f841706c700 1 Processor -- start 2023-06-23T01:04:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.183+0000 7f841706c700 1 -- start start 2023-06-23T01:04:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f841706c700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8410156ea0 0x7f8410157270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f841706c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f841015dc30 0x7f8410159c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f841706c700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f841015a190 0x7f84100a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f841706c700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841015d7a0 con 0x7f841015a190 2023-06-23T01:04:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f841706c700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841015a780 con 0x7f8410156ea0 2023-06-23T01:04:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f841706c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841015aa60 con 0x7f841015dc30 2023-06-23T01:04:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f8415609700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f841015a190 0x7f84100a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f8415609700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f841015a190 0x7f84100a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33948/0 (socket says 172.21.15.5:33948) 2023-06-23T01:04:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.184+0000 7f8415609700 1 -- 172.21.15.5:0/1742316195 learned_addr learned my addr 172.21.15.5:0/1742316195 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f8414e08700 1 --2- 172.21.15.5:0/1742316195 >> v2:172.21.15.152:3300/0 conn(0x7f8410156ea0 0x7f8410157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f840ffff700 1 --2- 172.21.15.5:0/1742316195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f841015dc30 0x7f8410159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f8415609700 1 -- 172.21.15.5:0/1742316195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f841015dc30 msgr2=0x7f8410159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f8415609700 1 --2- 172.21.15.5:0/1742316195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f841015dc30 0x7f8410159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f8415609700 1 -- 172.21.15.5:0/1742316195 >> v2:172.21.15.152:3300/0 conn(0x7f8410156ea0 msgr2=0x7f8410157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f8415609700 1 --2- 172.21.15.5:0/1742316195 >> v2:172.21.15.152:3300/0 conn(0x7f8410156ea0 0x7f8410157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f8415609700 1 -- 172.21.15.5:0/1742316195 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841015d430 con 0x7f841015a190 2023-06-23T01:04:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f8415609700 1 --2- 172.21.15.5:0/1742316195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100a9890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84100cd7b0 tx=0x7f8408009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934d3c6a9a4058ec server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f840f7fe700 1 -- 172.21.15.5:0/1742316195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800e070 con 0x7f841015a190 2023-06-23T01:04:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f840f7fe700 1 -- 172.21.15.5:0/1742316195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840800ba50 con 0x7f841015a190 2023-06-23T01:04:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f840f7fe700 1 -- 172.21.15.5:0/1742316195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080092e0 con 0x7f841015a190 2023-06-23T01:04:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f841706c700 1 -- 172.21.15.5:0/1742316195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 msgr2=0x7f84100a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f841706c700 1 --2- 172.21.15.5:0/1742316195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100a9890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f84100cd7b0 tx=0x7f8408009840 comp rx=0 tx=0).stop 2023-06-23T01:04:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.185+0000 7f841706c700 1 -- 172.21.15.5:0/1742316195 shutdown_connections 2023-06-23T01:04:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.186+0000 7f841706c700 1 --2- 172.21.15.5:0/1742316195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100a9890 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.186+0000 7f841706c700 1 --2- 172.21.15.5:0/1742316195 >> v2:172.21.15.152:3300/0 conn(0x7f8410156ea0 0x7f8410157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.186+0000 7f841706c700 1 --2- 172.21.15.5:0/1742316195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f841015dc30 0x7f8410159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.186+0000 7f841706c700 1 -- 172.21.15.5:0/1742316195 >> 172.21.15.5:0/1742316195 conn(0x7f841006e830 msgr2=0x7f841006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.186+0000 7f841706c700 1 -- 172.21.15.5:0/1742316195 shutdown_connections 2023-06-23T01:04:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.186+0000 7f841706c700 1 -- 172.21.15.5:0/1742316195 wait complete. 2023-06-23T01:04:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.187+0000 7f841706c700 1 Processor -- start 2023-06-23T01:04:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.187+0000 7f841706c700 1 -- start start 2023-06-23T01:04:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f841706c700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8410156ea0 0x7f84101565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f841706c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f841706c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841015dc30 0x7f84100ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f841706c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100b0e70 con 0x7f841015a190 2023-06-23T01:04:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f841706c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100afdd0 con 0x7f841015dc30 2023-06-23T01:04:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f841706c700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100b00b0 con 0x7f8410156ea0 2023-06-23T01:04:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33952/0 (socket says 172.21.15.5:33952) 2023-06-23T01:04:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 -- 172.21.15.5:0/1529070425 learned_addr learned my addr 172.21.15.5:0/1529070425 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f8415609700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841015dc30 0x7f84100ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f8414e08700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8410156ea0 0x7f84101565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 -- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8410156ea0 msgr2=0x7f84101565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8410156ea0 0x7f84101565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 -- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841015dc30 msgr2=0x7f84100ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841015dc30 0x7f84100ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 -- 172.21.15.5:0/1529070425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408009ae0 con 0x7f841015a190 2023-06-23T01:04:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840ffff700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100af510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d890 tx=0x7f83f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840cff9700 1 -- 172.21.15.5:0/1529070425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f80125f0 con 0x7f841015a190 2023-06-23T01:04:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840cff9700 1 -- 172.21.15.5:0/1529070425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f8012750 con 0x7f841015a190 2023-06-23T01:04:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84100b0390 con 0x7f841015a190 2023-06-23T01:04:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f840cff9700 1 -- 172.21.15.5:0/1529070425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8012af0 con 0x7f841015a190 2023-06-23T01:04:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.188+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84100b0950 con 0x7f841015a190 2023-06-23T01:04:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.189+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841004c1b0 con 0x7f841015a190 2023-06-23T01:04:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.190+0000 7f840cff9700 1 -- 172.21.15.5:0/1529070425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f83f801e460 con 0x7f841015a190 2023-06-23T01:04:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.190+0000 7f840cff9700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f83f0066ec0 0x7f83f0069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.190+0000 7f840cff9700 1 -- 172.21.15.5:0/1529070425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f83f8084aa0 con 0x7f841015a190 2023-06-23T01:04:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.190+0000 7f8414e08700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f83f0066ec0 0x7f83f0069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.191+0000 7f8414e08700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f83f0066ec0 0x7f83f0069370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8400006fd0 tx=0x7f8400008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.194+0000 7f840cff9700 1 -- 172.21.15.5:0/1529070425 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f83f8051c70 con 0x7f841015a190 2023-06-23T01:04:48.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.392+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84100b0b70 con 0x7f841015a190 2023-06-23T01:04:48.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.393+0000 7f840cff9700 1 -- 172.21.15.5:0/1529070425 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16831 (secure 0 0 0) 0x7f83f8055290 con 0x7f841015a190 2023-06-23T01:04:48.397 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:04:48.397 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"002f2efe-a784-4dcf-91c2-b9f520cd2c9c","created":"2023-06-23T01:04:28.422097+0000","modified":"2023-06-23T01:04:48.026130+0000","last_up_change":"2023-06-23T01:04:39.254387+0000","last_in_change":"2023-06-23T01:04:31.676217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:04:40.075004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T01:04:40.525837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T01:04:45.675725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T01:04:47.002674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2499edeb-6ba9-4392-b867-6a90f953982a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":87985},{"type":"v1","addr":"172.21.15.5:6817","nonce":87985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":87985},{"type":"v1","addr":"172.21.15.5:6819","nonce":87985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":87985},{"type":"v1","addr":"172.21.15.5:6826","nonce":87985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":87985},{"type":"v1","addr":"172.21.15.5:6822","nonce":87985}]},"public_addr":"172.21.15.5:6817/87985","cluster_addr":"172.21.15.5:6819/87985","heartbeat_back_addr":"172.21.15.5:6826/87985","heartbeat_front_addr":"172.21.15.5:6822/87985","state":["exists","up"]},{"osd":1,"uuid":"f2aa5a85-0aea-4bef-b45a-78297c1cd686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":87988},{"type":"v1","addr":"172.21.15.5:6801","nonce":87988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":87988},{"type":"v1","addr":"172.21.15.5:6804","nonce":87988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":87988},{"type":"v1","addr":"172.21.15.5:6812","nonce":87988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":87988},{"type":"v1","addr":"172.21.15.5:6808","nonce":87988}]},"public_addr":"172.21.15.5:6801/87988","cluster_addr":"172.21.15.5:6804/87988","heartbeat_back_addr":"172.21.15.5:6812/87988","heartbeat_front_addr":"172.21.15.5:6808/87988","state":["exists","up"]},{"osd":2,"uuid":"72aa25ff-c758-4780-9b74-fc1c316a3264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6821","nonce":87990},{"type":"v1","addr":"172.21.15.5:6823","nonce":87990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6825","nonce":87990},{"type":"v1","addr":"172.21.15.5:6827","nonce":87990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":87990},{"type":"v1","addr":"172.21.15.5:6831","nonce":87990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":87990},{"type":"v1","addr":"172.21.15.5:6829","nonce":87990}]},"public_addr":"172.21.15.5:6823/87990","cluster_addr":"172.21.15.5:6827/87990","heartbeat_back_addr":"172.21.15.5:6831/87990","heartbeat_front_addr":"172.21.15.5:6829/87990","state":["exists","up"]},{"osd":3,"uuid":"77dd6974-49b0-4288-b310-e7a0729ea2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":87991},{"type":"v1","addr":"172.21.15.5:6805","nonce":87991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":87991},{"type":"v1","addr":"172.21.15.5:6809","nonce":87991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":87991},{"type":"v1","addr":"172.21.15.5:6815","nonce":87991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":87991},{"type":"v1","addr":"172.21.15.5:6813","nonce":87991}]},"public_addr":"172.21.15.5:6805/87991","cluster_addr":"172.21.15.5:6809/87991","heartbeat_back_addr":"172.21.15.5:6815/87991","heartbeat_front_addr":"172.21.15.5:6813/87991","state":["exists","up"]},{"osd":4,"uuid":"80562fc5-676d-4cee-9978-4a94fca8df84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":88231},{"type":"v1","addr":"172.21.15.152:6817","nonce":88231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":88231},{"type":"v1","addr":"172.21.15.152:6819","nonce":88231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":88231},{"type":"v1","addr":"172.21.15.152:6823","nonce":88231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":88231},{"type":"v1","addr":"172.21.15.152:6821","nonce":88231}]},"public_addr":"172.21.15.152:6817/88231","cluster_addr":"172.21.15.152:6819/88231","heartbeat_back_addr":"172.21.15.152:6823/88231","heartbeat_front_addr":"172.21.15.152:6821/88231","state":["exists","up"]},{"osd":5,"uuid":"1db79ad4-997e-4516-a40b-90ee44d4957a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":88215},{"type":"v1","addr":"172.21.15.152:6801","nonce":88215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":88215},{"type":"v1","addr":"172.21.15.152:6803","nonce":88215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":88215},{"type":"v1","addr":"172.21.15.152:6807","nonce":88215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":88215},{"type":"v1","addr":"172.21.15.152:6805","nonce":88215}]},"public_addr":"172.21.15.152:6801/88215","cluster_addr":"172.21.15.152:6803/88215","heartbeat_back_addr":"172.21.15.152:6807/88215","heartbeat_front_addr":"172.21.15.152:6805/88215","state":["exists","up"]},{"osd":6,"uuid":"c18d6095-c8f0-4d76-94a8-f8de40a45195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":88216},{"type":"v1","addr":"172.21.15.152:6809","nonce":88216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":88216},{"type":"v1","addr":"172.21.15.152:6811","nonce":88216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":88216},{"type":"v1","addr":"172.21.15.152:6815","nonce":88216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":88216},{"type":"v1","addr":"172.21.15.152:6813","nonce":88216}]},"public_addr":"172.21.15.152:6809/88216","cluster_addr":"172.21.15.152:6811/88216","heartbeat_back_addr":"172.21.15.152:6815/88216","heartbeat_front_addr":"172.21.15.152:6813/88216","state":["exists","up"]},{"osd":7,"uuid":"6ade832e-0e46-4c4f-8fb2-3a86a643e19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":88246},{"type":"v1","addr":"172.21.15.152:6825","nonce":88246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":88246},{"type":"v1","addr":"172.21.15.152:6827","nonce":88246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":88246},{"type":"v1","addr":"172.21.15.152:6831","nonce":88246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":88246},{"type":"v1","addr":"172.21.15.152:6829","nonce":88246}]},"public_addr":"172.21.15.152:6825/88246","cluster_addr":"172.21.15.152:6827/88246","heartbeat_back_addr":"172.21.15.152:6831/88246","heartbeat_front_addr":"172.21.15.152:6829/88246","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.831143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.816429+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.901041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.813091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.819606+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.865581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.822019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.825405+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f83f0066ec0 msgr2=0x7f83f0069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f83f0066ec0 0x7f83f0069370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8400006fd0 tx=0x7f8400008040 comp rx=0 tx=0).stop 2023-06-23T01:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 msgr2=0x7f84100af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100af510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83f800d890 tx=0x7f83f8008040 comp rx=0 tx=0).stop 2023-06-23T01:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 shutdown_connections 2023-06-23T01:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f83f0066ec0 0x7f83f0069370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f841015dc30 0x7f84100ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8410156ea0 0x7f84101565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 --2- 172.21.15.5:0/1529070425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f841015a190 0x7f84100af510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.395+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 >> 172.21.15.5:0/1529070425 conn(0x7f841006e830 msgr2=0x7f841006ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.396+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 shutdown_connections 2023-06-23T01:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.396+0000 7f841706c700 1 -- 172.21.15.5:0/1529070425 wait complete. 2023-06-23T01:04:48.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T01:04:48.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.526+0000 7fe6d2f14700 1 Processor -- start 2023-06-23T01:04:48.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.526+0000 7fe6d2f14700 1 -- start start 2023-06-23T01:04:48.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d2f14700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6cc0b2550 0x7fe6cc15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d2f14700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0b2920 0x7fe6cc06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d2f14700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc15cf70 con 0x7fe6cc0cede0 2023-06-23T01:04:48.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d2f14700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc0cb470 con 0x7fe6cc0b2550 2023-06-23T01:04:48.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d2f14700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0cb750 con 0x7fe6cc0b2920 2023-06-23T01:04:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d0cb0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6cc0b2550 0x7fe6cc15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d0cb0700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6cc0b2550 0x7fe6cc15d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33960/0 (socket says 172.21.15.5:33960) 2023-06-23T01:04:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d0cb0700 1 -- 172.21.15.5:0/869727763 learned_addr learned my addr 172.21.15.5:0/869727763 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d14b1700 1 --2- 172.21.15.5:0/869727763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0b2920 0x7fe6cc06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d0cb0700 1 -- 172.21.15.5:0/869727763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0b2920 msgr2=0x7fe6cc06ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d0cb0700 1 --2- 172.21.15.5:0/869727763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0b2920 0x7fe6cc06ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d0cb0700 1 -- 172.21.15.5:0/869727763 >> v1:172.21.15.152:6789/0 conn(0x7fe6cc0cede0 legacy=0x7fe6cc0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d0cb0700 1 -- 172.21.15.5:0/869727763 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc15d1c0 con 0x7fe6cc0b2550 2023-06-23T01:04:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d14b1700 1 --2- 172.21.15.5:0/869727763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0b2920 0x7fe6cc06ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.527+0000 7fe6d0cb0700 1 --2- 172.21.15.5:0/869727763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc15d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00099b0 tx=0x7fe6c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d10aa6aee147f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6cb7fe700 1 -- 172.21.15.5:0/869727763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000b860 con 0x7fe6cc0b2550 2023-06-23T01:04:48.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6cb7fe700 1 -- 172.21.15.5:0/869727763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c000b9c0 con 0x7fe6cc0b2550 2023-06-23T01:04:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6cb7fe700 1 -- 172.21.15.5:0/869727763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000bd60 con 0x7fe6cc0b2550 2023-06-23T01:04:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6d2f14700 1 -- 172.21.15.5:0/869727763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 msgr2=0x7fe6cc15d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/869727763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc15d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00099b0 tx=0x7fe6c0009250 comp rx=0 tx=0).stop 2023-06-23T01:04:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6d2f14700 1 -- 172.21.15.5:0/869727763 shutdown_connections 2023-06-23T01:04:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/869727763 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0b2920 0x7fe6cc06ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/869727763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc15d9c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6d2f14700 1 -- 172.21.15.5:0/869727763 >> 172.21.15.5:0/869727763 conn(0x7fe6cc0d02a0 msgr2=0x7fe6cc0d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:48.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6d2f14700 1 -- 172.21.15.5:0/869727763 shutdown_connections 2023-06-23T01:04:48.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.528+0000 7fe6d2f14700 1 -- 172.21.15.5:0/869727763 wait complete. 2023-06-23T01:04:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.529+0000 7fe6d2f14700 1 Processor -- start 2023-06-23T01:04:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.529+0000 7fe6d2f14700 1 -- start start 2023-06-23T01:04:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d2f14700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d2f14700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6cc0b2920 0x7fe6cc0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d2f14700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0cede0 0x7fe6cc0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d2f14700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc156a70 con 0x7fe6cc0b2550 2023-06-23T01:04:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d2f14700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc156f00 con 0x7fe6cc0b2920 2023-06-23T01:04:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d2f14700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc152de0 con 0x7fe6cc0cede0 2023-06-23T01:04:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc1528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33966/0 (socket says 172.21.15.5:33966) 2023-06-23T01:04:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 -- 172.21.15.5:0/2779580951 learned_addr learned my addr 172.21.15.5:0/2779580951 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6cbfff700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6cc0b2920 0x7fe6cc0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d14b1700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0cede0 0x7fe6cc0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 -- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0cede0 msgr2=0x7fe6cc0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0cede0 0x7fe6cc0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 -- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6cc0b2920 msgr2=0x7fe6cc0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6cc0b2920 0x7fe6cc0c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 -- 172.21.15.5:0/2779580951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00094f0 con 0x7fe6cc0b2550 2023-06-23T01:04:48.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6cbfff700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6cc0b2920 0x7fe6cc0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:48.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d14b1700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0cede0 0x7fe6cc0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T01:04:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.530+0000 7fe6d0cb0700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc1528a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009290 tx=0x7fe6c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.531+0000 7fe6c8ff9700 1 -- 172.21.15.5:0/2779580951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0024440 con 0x7fe6cc0b2550 2023-06-23T01:04:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.531+0000 7fe6c8ff9700 1 -- 172.21.15.5:0/2779580951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c00245a0 con 0x7fe6cc0b2550 2023-06-23T01:04:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.531+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc155e60 con 0x7fe6cc0b2550 2023-06-23T01:04:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.531+0000 7fe6c8ff9700 1 -- 172.21.15.5:0/2779580951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0024940 con 0x7fe6cc0b2550 2023-06-23T01:04:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.531+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc156420 con 0x7fe6cc0b2550 2023-06-23T01:04:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.532+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc04c1b0 con 0x7fe6cc0b2550 2023-06-23T01:04:48.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.532+0000 7fe6c8ff9700 1 -- 172.21.15.5:0/2779580951 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7fe6c001a420 con 0x7fe6cc0b2550 2023-06-23T01:04:48.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.533+0000 7fe6c8ff9700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fe6b0066e90 0x7fe6b0069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.533+0000 7fe6c8ff9700 1 -- 172.21.15.5:0/2779580951 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe6c0086410 con 0x7fe6cc0b2550 2023-06-23T01:04:48.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.533+0000 7fe6cbfff700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fe6b0066e90 0x7fe6b0069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.533+0000 7fe6cbfff700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fe6b0066e90 0x7fe6b0069340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0c7400 tx=0x7fe6b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.537+0000 7fe6c8ff9700 1 -- 172.21.15.5:0/2779580951 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7fe6c00535e0 con 0x7fe6cc0b2550 2023-06-23T01:04:48.611 INFO:tasks.ceph.mds.b.smithi152.stderr:2023-06-23T01:04:48.608+0000 7fc536b5e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T01:04:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.733+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6cc0ce020 con 0x7fe6cc0b2550 2023-06-23T01:04:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.734+0000 7fe6c8ff9700 1 -- 172.21.15.5:0/2779580951 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16831 (secure 0 0 0) 0x7fe6c0056c00 con 0x7fe6cc0b2550 2023-06-23T01:04:48.737 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-06-23T01:04:48.737 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"002f2efe-a784-4dcf-91c2-b9f520cd2c9c","created":"2023-06-23T01:04:28.422097+0000","modified":"2023-06-23T01:04:48.026130+0000","last_up_change":"2023-06-23T01:04:39.254387+0000","last_in_change":"2023-06-23T01:04:31.676217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T01:04:40.075004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-23T01:04:40.525837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-23T01:04:45.675725+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-23T01:04:47.002674+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2499edeb-6ba9-4392-b867-6a90f953982a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":87985},{"type":"v1","addr":"172.21.15.5:6817","nonce":87985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":87985},{"type":"v1","addr":"172.21.15.5:6819","nonce":87985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":87985},{"type":"v1","addr":"172.21.15.5:6826","nonce":87985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":87985},{"type":"v1","addr":"172.21.15.5:6822","nonce":87985}]},"public_addr":"172.21.15.5:6817/87985","cluster_addr":"172.21.15.5:6819/87985","heartbeat_back_addr":"172.21.15.5:6826/87985","heartbeat_front_addr":"172.21.15.5:6822/87985","state":["exists","up"]},{"osd":1,"uuid":"f2aa5a85-0aea-4bef-b45a-78297c1cd686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":87988},{"type":"v1","addr":"172.21.15.5:6801","nonce":87988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":87988},{"type":"v1","addr":"172.21.15.5:6804","nonce":87988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":87988},{"type":"v1","addr":"172.21.15.5:6812","nonce":87988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":87988},{"type":"v1","addr":"172.21.15.5:6808","nonce":87988}]},"public_addr":"172.21.15.5:6801/87988","cluster_addr":"172.21.15.5:6804/87988","heartbeat_back_addr":"172.21.15.5:6812/87988","heartbeat_front_addr":"172.21.15.5:6808/87988","state":["exists","up"]},{"osd":2,"uuid":"72aa25ff-c758-4780-9b74-fc1c316a3264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6821","nonce":87990},{"type":"v1","addr":"172.21.15.5:6823","nonce":87990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6825","nonce":87990},{"type":"v1","addr":"172.21.15.5:6827","nonce":87990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":87990},{"type":"v1","addr":"172.21.15.5:6831","nonce":87990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":87990},{"type":"v1","addr":"172.21.15.5:6829","nonce":87990}]},"public_addr":"172.21.15.5:6823/87990","cluster_addr":"172.21.15.5:6827/87990","heartbeat_back_addr":"172.21.15.5:6831/87990","heartbeat_front_addr":"172.21.15.5:6829/87990","state":["exists","up"]},{"osd":3,"uuid":"77dd6974-49b0-4288-b310-e7a0729ea2ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":87991},{"type":"v1","addr":"172.21.15.5:6805","nonce":87991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":87991},{"type":"v1","addr":"172.21.15.5:6809","nonce":87991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":87991},{"type":"v1","addr":"172.21.15.5:6815","nonce":87991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":87991},{"type":"v1","addr":"172.21.15.5:6813","nonce":87991}]},"public_addr":"172.21.15.5:6805/87991","cluster_addr":"172.21.15.5:6809/87991","heartbeat_back_addr":"172.21.15.5:6815/87991","heartbeat_front_addr":"172.21.15.5:6813/87991","state":["exists","up"]},{"osd":4,"uuid":"80562fc5-676d-4cee-9978-4a94fca8df84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":88231},{"type":"v1","addr":"172.21.15.152:6817","nonce":88231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":88231},{"type":"v1","addr":"172.21.15.152:6819","nonce":88231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":88231},{"type":"v1","addr":"172.21.15.152:6823","nonce":88231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":88231},{"type":"v1","addr":"172.21.15.152:6821","nonce":88231}]},"public_addr":"172.21.15.152:6817/88231","cluster_addr":"172.21.15.152:6819/88231","heartbeat_back_addr":"172.21.15.152:6823/88231","heartbeat_front_addr":"172.21.15.152:6821/88231","state":["exists","up"]},{"osd":5,"uuid":"1db79ad4-997e-4516-a40b-90ee44d4957a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":88215},{"type":"v1","addr":"172.21.15.152:6801","nonce":88215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":88215},{"type":"v1","addr":"172.21.15.152:6803","nonce":88215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":88215},{"type":"v1","addr":"172.21.15.152:6807","nonce":88215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":88215},{"type":"v1","addr":"172.21.15.152:6805","nonce":88215}]},"public_addr":"172.21.15.152:6801/88215","cluster_addr":"172.21.15.152:6803/88215","heartbeat_back_addr":"172.21.15.152:6807/88215","heartbeat_front_addr":"172.21.15.152:6805/88215","state":["exists","up"]},{"osd":6,"uuid":"c18d6095-c8f0-4d76-94a8-f8de40a45195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":88216},{"type":"v1","addr":"172.21.15.152:6809","nonce":88216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":88216},{"type":"v1","addr":"172.21.15.152:6811","nonce":88216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":88216},{"type":"v1","addr":"172.21.15.152:6815","nonce":88216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":88216},{"type":"v1","addr":"172.21.15.152:6813","nonce":88216}]},"public_addr":"172.21.15.152:6809/88216","cluster_addr":"172.21.15.152:6811/88216","heartbeat_back_addr":"172.21.15.152:6815/88216","heartbeat_front_addr":"172.21.15.152:6813/88216","state":["exists","up"]},{"osd":7,"uuid":"6ade832e-0e46-4c4f-8fb2-3a86a643e19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":88246},{"type":"v1","addr":"172.21.15.152:6825","nonce":88246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":88246},{"type":"v1","addr":"172.21.15.152:6827","nonce":88246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":88246},{"type":"v1","addr":"172.21.15.152:6831","nonce":88246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":88246},{"type":"v1","addr":"172.21.15.152:6829","nonce":88246}]},"public_addr":"172.21.15.152:6825/88246","cluster_addr":"172.21.15.152:6827/88246","heartbeat_back_addr":"172.21.15.152:6831/88246","heartbeat_front_addr":"172.21.15.152:6829/88246","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.831143+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.816429+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.901041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.813091+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.819606+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.865581+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.822019+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T01:04:36.825405+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T01:04:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fe6b0066e90 msgr2=0x7fe6b0069340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fe6b0066e90 0x7fe6b0069340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0c7400 tx=0x7fe6b8009040 comp rx=0 tx=0).stop 2023-06-23T01:04:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 msgr2=0x7fe6cc1528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc1528a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009290 tx=0x7fe6c0017000 comp rx=0 tx=0).stop 2023-06-23T01:04:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 shutdown_connections 2023-06-23T01:04:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7fe6b0066e90 0x7fe6b0069340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe6cc0cede0 0x7fe6cc0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc0b2550 0x7fe6cc1528a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 --2- 172.21.15.5:0/2779580951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6cc0b2920 0x7fe6cc0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.735+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 >> 172.21.15.5:0/2779580951 conn(0x7fe6cc0d02a0 msgr2=0x7fe6cc15e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.736+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 shutdown_connections 2023-06-23T01:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.736+0000 7fe6d2f14700 1 -- 172.21.15.5:0/2779580951 wait complete. 2023-06-23T01:04:48.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-23T01:04:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.869+0000 7f270643b700 1 Processor -- start 2023-06-23T01:04:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.869+0000 7f270643b700 1 -- start start 2023-06-23T01:04:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.870+0000 7f270643b700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2700159740 0x7f270015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.870+0000 7f270643b700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f270006c000 0x7f27000a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.870+0000 7f270643b700 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270006a720 con 0x7f27000a9dd0 2023-06-23T01:04:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.870+0000 7f270643b700 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270015e2f0 con 0x7f2700159740 2023-06-23T01:04:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.870+0000 7f270643b700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270006a3f0 con 0x7f270006c000 2023-06-23T01:04:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.870+0000 7f26fffff700 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2700159740 0x7f270015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.870+0000 7f27049d8700 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f27000a9dd0 0x7f270006abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45908/0 (socket says 172.21.15.5:45908) 2023-06-23T01:04:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.870+0000 7f27049d8700 1 -- 172.21.15.5:0/3557914290 learned_addr learned my addr 172.21.15.5:0/3557914290 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-06-23T01:04:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26ff7fe700 1 --2- 172.21.15.5:0/3557914290 >> v2:172.21.15.152:3300/0 conn(0x7f270006c000 0x7f27000a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26fffff700 1 -- 172.21.15.5:0/3557914290 >> v2:172.21.15.152:3300/0 conn(0x7f270006c000 msgr2=0x7f27000a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26fffff700 1 --2- 172.21.15.5:0/3557914290 >> v2:172.21.15.152:3300/0 conn(0x7f270006c000 0x7f27000a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26fffff700 1 -- 172.21.15.5:0/3557914290 >> v1:172.21.15.5:6789/0 conn(0x7f27000a9dd0 legacy=0x7f270006abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26fffff700 1 -- 172.21.15.5:0/3557914290 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270006ce60 con 0x7f2700159740 2023-06-23T01:04:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26feffd700 1 -- 172.21.15.5:0/3557914290 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794108773 0 0) 0x7f270006a720 con 0x7f27000a9dd0 2023-06-23T01:04:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26fffff700 1 --2- 172.21.15.5:0/3557914290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2700159740 0x7f270015ddb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27000ccfa0 tx=0x7f26f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e62d2505a31642f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26feffd700 1 -- 172.21.15.5:0/3557914290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000b6b0 con 0x7f2700159740 2023-06-23T01:04:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26feffd700 1 -- 172.21.15.5:0/3557914290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f000b810 con 0x7f2700159740 2023-06-23T01:04:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f26feffd700 1 -- 172.21.15.5:0/3557914290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000bbb0 con 0x7f2700159740 2023-06-23T01:04:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f270643b700 1 -- 172.21.15.5:0/3557914290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2700159740 msgr2=0x7f270015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f270643b700 1 --2- 172.21.15.5:0/3557914290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2700159740 0x7f270015ddb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27000ccfa0 tx=0x7f26f00091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f270643b700 1 -- 172.21.15.5:0/3557914290 shutdown_connections 2023-06-23T01:04:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f270643b700 1 --2- 172.21.15.5:0/3557914290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2700159740 0x7f270015ddb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f270643b700 1 --2- 172.21.15.5:0/3557914290 >> v2:172.21.15.152:3300/0 conn(0x7f270006c000 0x7f27000a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.871+0000 7f270643b700 1 -- 172.21.15.5:0/3557914290 >> 172.21.15.5:0/3557914290 conn(0x7f27000ccfd0 msgr2=0x7f27000cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.872+0000 7f270643b700 1 -- 172.21.15.5:0/3557914290 shutdown_connections 2023-06-23T01:04:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.872+0000 7f270643b700 1 -- 172.21.15.5:0/3557914290 wait complete. 2023-06-23T01:04:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.873+0000 7f270643b700 1 Processor -- start 2023-06-23T01:04:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.873+0000 7f270643b700 1 -- start start 2023-06-23T01:04:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.873+0000 7f270643b700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f270006c000 0x7f2700169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f270643b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27000a9dd0 0x7f27001579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f270643b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2700159740 0x7f270016a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f270643b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270016a230 con 0x7f27000a9dd0 2023-06-23T01:04:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f270643b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270015d190 con 0x7f2700159740 2023-06-23T01:04:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f270643b700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2700157f30 con 0x7f270006c000 2023-06-23T01:04:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27000a9dd0 0x7f27001579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27000a9dd0 0x7f27001579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33996/0 (socket says 172.21.15.5:33996) 2023-06-23T01:04:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 -- 172.21.15.5:0/2418290665 learned_addr learned my addr 172.21.15.5:0/2418290665 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f27049d8700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2700159740 0x7f270016a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26fffff700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f270006c000 0x7f2700169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 -- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f270006c000 msgr2=0x7f2700169cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f270006c000 0x7f2700169cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 -- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2700159740 msgr2=0x7f270016a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2700159740 0x7f270016a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 -- 172.21.15.5:0/2418290665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0009420 con 0x7f27000a9dd0 2023-06-23T01:04:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f27049d8700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2700159740 0x7f270016a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:04:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26fffff700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f270006c000 0x7f2700169cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:04:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26ff7fe700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27000a9dd0 0x7f27001579f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c800 tx=0x7f26e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26f7fff700 1 -- 172.21.15.5:0/2418290665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8011600 con 0x7f27000a9dd0 2023-06-23T01:04:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26f7fff700 1 -- 172.21.15.5:0/2418290665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e8011760 con 0x7f27000a9dd0 2023-06-23T01:04:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f26f7fff700 1 -- 172.21.15.5:0/2418290665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8011b20 con 0x7f27000a9dd0 2023-06-23T01:04:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f270643b700 1 -- 172.21.15.5:0/2418290665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270015a750 con 0x7f27000a9dd0 2023-06-23T01:04:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.874+0000 7f270643b700 1 -- 172.21.15.5:0/2418290665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270015c860 con 0x7f27000a9dd0 2023-06-23T01:04:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.875+0000 7f270643b700 1 -- 172.21.15.5:0/2418290665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270004c1b0 con 0x7f27000a9dd0 2023-06-23T01:04:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.876+0000 7f26f7fff700 1 -- 172.21.15.5:0/2418290665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f26e80193f0 con 0x7f27000a9dd0 2023-06-23T01:04:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.876+0000 7f26f7fff700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f26e0066e90 0x7f26e0069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.876+0000 7f26f7fff700 1 -- 172.21.15.5:0/2418290665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f26e80849d0 con 0x7f27000a9dd0 2023-06-23T01:04:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.877+0000 7f26fffff700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f26e0066e90 0x7f26e0069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.877+0000 7f26fffff700 1 --2- 172.21.15.5:0/2418290665 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f26e0066e90 0x7f26e0069340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26f00091e0 tx=0x7f26f0017000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:48.881+0000 7f26f7fff700 1 -- 172.21.15.5:0/2418290665 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f26e8051ba0 con 0x7f27000a9dd0 2023-06-23T01:04:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 1 closest matches: 2023-06-23T01:04:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-23T01:04:49.125 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2023-06-23T01:04:49.127 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-23T01:04:49.128 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-23T01:04:49.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-23T01:04:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.247+0000 7f81ba74f700 1 Processor -- start 2023-06-23T01:04:49.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.247+0000 7f81ba74f700 1 -- start start 2023-06-23T01:04:49.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.248+0000 7f81ba74f700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f81b40b2550 0x7f81b415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.248+0000 7f81ba74f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40b2920 0x7f81b406ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.248+0000 7f81ba74f700 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b415cf70 con 0x7f81b40cede0 2023-06-23T01:04:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.248+0000 7f81ba74f700 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b40cb470 con 0x7f81b40b2550 2023-06-23T01:04:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.248+0000 7f81ba74f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40cb750 con 0x7f81b40b2920 2023-06-23T01:04:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b8cec700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40b2920 0x7f81b406ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b3fff700 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f81b40b2550 0x7f81b415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:49.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b8cec700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40b2920 0x7f81b406ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.5:37244/0 (socket says 172.21.15.5:37244) 2023-06-23T01:04:49.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b8cec700 1 -- 172.21.15.5:0/1461557861 learned_addr learned my addr 172.21.15.5:0/1461557861 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:49.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b3fff700 1 -- 172.21.15.5:0/1461557861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40b2920 msgr2=0x7f81b406ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T01:04:49.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b3fff700 1 --2- 172.21.15.5:0/1461557861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40b2920 0x7f81b406ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:49.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b3fff700 1 -- 172.21.15.5:0/1461557861 >> v1:172.21.15.152:6789/0 conn(0x7f81b40cede0 legacy=0x7f81b40ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:49.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b3fff700 1 -- 172.21.15.5:0/1461557861 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b415d1c0 con 0x7f81b40b2550 2023-06-23T01:04:49.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b2ffd700 1 -- 172.21.15.5:0/1461557861 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932523750 0 0) 0x7f81b415cf70 con 0x7f81b40cede0 2023-06-23T01:04:49.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b8cec700 1 --2- 172.21.15.5:0/1461557861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40b2920 0x7f81b406ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T01:04:49.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b3fff700 1 --2- 172.21.15.5:0/1461557861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2550 0x7f81b415d9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81a80098e0 tx=0x7f81a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e848170288a0ae9f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:49.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b2ffd700 1 -- 172.21.15.5:0/1461557861 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800b640 con 0x7f81b40b2550 2023-06-23T01:04:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b2ffd700 1 -- 172.21.15.5:0/1461557861 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a800b7a0 con 0x7f81b40b2550 2023-06-23T01:04:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.249+0000 7f81b2ffd700 1 -- 172.21.15.5:0/1461557861 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800bb40 con 0x7f81b40b2550 2023-06-23T01:04:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.250+0000 7f81ba74f700 1 -- 172.21.15.5:0/1461557861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2550 msgr2=0x7f81b415d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:49.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.250+0000 7f81ba74f700 1 --2- 172.21.15.5:0/1461557861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2550 0x7f81b415d9c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81a80098e0 tx=0x7f81a80091a0 comp rx=0 tx=0).stop 2023-06-23T01:04:49.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.250+0000 7f81ba74f700 1 -- 172.21.15.5:0/1461557861 shutdown_connections 2023-06-23T01:04:49.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.250+0000 7f81ba74f700 1 --2- 172.21.15.5:0/1461557861 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40b2920 0x7f81b406ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:49.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.250+0000 7f81ba74f700 1 --2- 172.21.15.5:0/1461557861 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2550 0x7f81b415d9c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:49.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.250+0000 7f81ba74f700 1 -- 172.21.15.5:0/1461557861 >> 172.21.15.5:0/1461557861 conn(0x7f81b40d02a0 msgr2=0x7f81b40d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T01:04:49.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.250+0000 7f81ba74f700 1 -- 172.21.15.5:0/1461557861 shutdown_connections 2023-06-23T01:04:49.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.250+0000 7f81ba74f700 1 -- 172.21.15.5:0/1461557861 wait complete. 2023-06-23T01:04:49.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.251+0000 7f81ba74f700 1 Processor -- start 2023-06-23T01:04:49.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.251+0000 7f81ba74f700 1 -- start start 2023-06-23T01:04:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81ba74f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b40b2550 0x7f81b415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81ba74f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2920 0x7f81b40c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:49.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81ba74f700 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40cede0 0x7f81b4153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81ba74f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40c79c0 con 0x7f81b40b2550 2023-06-23T01:04:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81ba74f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40c6ad0 con 0x7f81b40b2920 2023-06-23T01:04:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81ba74f700 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b40c7c80 con 0x7f81b40cede0 2023-06-23T01:04:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b40b2550 0x7f81b415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b40b2550 0x7f81b415f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34008/0 (socket says 172.21.15.5:34008) 2023-06-23T01:04:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b3fff700 1 -- 172.21.15.5:0/1432673733 learned_addr learned my addr 172.21.15.5:0/1432673733 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-06-23T01:04:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b8cec700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40cede0 0x7f81b4153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b37fe700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2920 0x7f81b40c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:49.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b3fff700 1 -- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40cede0 msgr2=0x7f81b4153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:49.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b3fff700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f81b40cede0 0x7f81b4153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:49.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b3fff700 1 -- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2920 msgr2=0x7f81b40c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T01:04:49.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b3fff700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2920 0x7f81b40c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T01:04:49.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b3fff700 1 -- 172.21.15.5:0/1432673733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac009ae0 con 0x7f81b40b2550 2023-06-23T01:04:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.252+0000 7f81b37fe700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f81b40b2920 0x7f81b40c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T01:04:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.253+0000 7f81b3fff700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81b40b2550 0x7f81b415f960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f81b406e820 tx=0x7f81a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.253+0000 7f819bfff700 1 -- 172.21.15.5:0/1432673733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8000d10 con 0x7f81b40b2550 2023-06-23T01:04:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.253+0000 7f819bfff700 1 -- 172.21.15.5:0/1432673733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a800bb20 con 0x7f81b40b2550 2023-06-23T01:04:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.253+0000 7f81ba74f700 1 -- 172.21.15.5:0/1432673733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a8009420 con 0x7f81b40b2550 2023-06-23T01:04:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.253+0000 7f819bfff700 1 -- 172.21.15.5:0/1432673733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8011650 con 0x7f81b40b2550 2023-06-23T01:04:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.253+0000 7f81ba74f700 1 -- 172.21.15.5:0/1432673733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b40c62e0 con 0x7f81b40b2550 2023-06-23T01:04:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.254+0000 7f81ba74f700 1 -- 172.21.15.5:0/1432673733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b404c1b0 con 0x7f81b40b2550 2023-06-23T01:04:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.254+0000 7f819bfff700 1 -- 172.21.15.5:0/1432673733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 84648+0+0 (secure 0 0 0) 0x7f81a8011a00 con 0x7f81b40b2550 2023-06-23T01:04:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.255+0000 7f819bfff700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f81a0066e90 0x7f81a0069340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T01:04:49.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.255+0000 7f819bfff700 1 -- 172.21.15.5:0/1432673733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f81a8085be0 con 0x7f81b40b2550 2023-06-23T01:04:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.255+0000 7f81b37fe700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f81a0066e90 0x7f81a0069340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T01:04:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.255+0000 7f81b37fe700 1 --2- 172.21.15.5:0/1432673733 >> [v2:172.21.15.152:6832/87615,v1:172.21.15.152:6833/87615] conn(0x7f81a0066e90 0x7f81a0069340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f81a4007fd0 tx=0x7f81a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T01:04:49.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-06-23T01:04:49.259+0000 7f819bfff700 1 -- 172.21.15.5:0/1432673733 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173437 (secure 0 0 0) 0x7f81a8052db0 con 0x7f81b40b2550 2023-06-23T01:04:49.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T01:04:49.427 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T01:04:49.504 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 1 closest matches: 2023-06-23T01:04:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:fs set |refuse_client_session|max_xattr_size [--yes-i-really-mean-it] [--yes-i-really-really-mean-it] 2023-06-23T01:04:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2023-06-23T01:04:49.507 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-06-23T01:04:49.508 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi005 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-23T01:04:49.509 INFO:teuthology.misc:Shutting down mds daemons... 2023-06-23T01:04:49.510 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-23T01:04:49.511 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.556 INFO:tasks.ceph.mds.a:Stopped 2023-06-23T01:04:49.557 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-23T01:04:49.558 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.606 INFO:tasks.ceph.mds.c:Stopped 2023-06-23T01:04:49.606 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-23T01:04:49.607 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.620 INFO:tasks.ceph.mds.b:Stopped 2023-06-23T01:04:49.622 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-23T01:04:49.623 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.672 INFO:tasks.ceph.mds.d:Stopped 2023-06-23T01:04:49.673 INFO:teuthology.misc:Shutting down osd daemons... 2023-06-23T01:04:49.674 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-06-23T01:04:49.675 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.701 INFO:tasks.ceph.osd.0:Stopped 2023-06-23T01:04:49.702 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-06-23T01:04:49.702 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.774 INFO:tasks.ceph.osd.1:Stopped 2023-06-23T01:04:49.775 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-06-23T01:04:49.775 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.837 INFO:tasks.ceph.osd.2:Stopped 2023-06-23T01:04:49.838 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-06-23T01:04:49.838 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.907 INFO:tasks.ceph.osd.3:Stopped 2023-06-23T01:04:49.907 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-06-23T01:04:49.908 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:49.947 INFO:tasks.ceph.osd.4:Stopped 2023-06-23T01:04:49.948 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-06-23T01:04:49.949 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:50.016 INFO:tasks.ceph.osd.5:Stopped 2023-06-23T01:04:50.017 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-06-23T01:04:50.017 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:50.089 INFO:tasks.ceph.osd.6:Stopped 2023-06-23T01:04:50.091 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-06-23T01:04:50.091 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:50.155 INFO:tasks.ceph.osd.7:Stopped 2023-06-23T01:04:50.156 INFO:teuthology.misc:Shutting down mgr daemons... 2023-06-23T01:04:50.156 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-06-23T01:04:50.157 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:50.179 INFO:tasks.ceph.mgr.y:Stopped 2023-06-23T01:04:50.180 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-06-23T01:04:50.180 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:50.222 INFO:tasks.ceph.mgr.x:Stopped 2023-06-23T01:04:50.223 INFO:teuthology.misc:Shutting down mon daemons... 2023-06-23T01:04:50.223 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-06-23T01:04:50.224 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:50.233 INFO:tasks.ceph.mon.a:Stopped 2023-06-23T01:04:50.234 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-06-23T01:04:50.234 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:50.273 INFO:tasks.ceph.mon.b:Stopped 2023-06-23T01:04:50.274 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-06-23T01:04:50.274 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T01:04:50.325 INFO:tasks.ceph.mon.c:Stopped 2023-06-23T01:04:50.325 INFO:tasks.ceph:Checking cluster log for badness... 2023-06-23T01:04:50.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-06-23T01:04:50.358 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:04:50.359 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-06-23T01:04:50.673 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:04:50.674 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-06-23T01:04:50.787 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:04:50.787 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-06-23T01:04:50.851 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:04:50.851 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-06-23T01:04:50.930 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:50.930 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-06-23T01:04:51.311 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:51.312 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-06-23T01:04:51.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:51.383 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-06-23T01:04:51.476 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:51.476 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-06-23T01:04:51.550 INFO:tasks.ceph:Archiving mon data... 2023-06-23T01:04:51.551 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/data/mon.a.tgz 2023-06-23T01:04:51.551 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2023-06-23T01:04:51.570 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.ztTnrewAfD 2023-06-23T01:04:51.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ztTnrewAfD 2023-06-23T01:04:51.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.ztTnrewAfD 2023-06-23T01:04:51.781 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.ztTnrewAfD is 196KB 2023-06-23T01:04:51.807 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.ztTnrewAfD 2023-06-23T01:04:51.822 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/data/mon.b.tgz 2023-06-23T01:04:51.823 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2023-06-23T01:04:51.839 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.PFAt3NSAFR 2023-06-23T01:04:51.839 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.PFAt3NSAFR 2023-06-23T01:04:51.952 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.PFAt3NSAFR 2023-06-23T01:04:52.034 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.PFAt3NSAFR is 271KB 2023-06-23T01:04:52.129 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.PFAt3NSAFR 2023-06-23T01:04:52.144 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/data/mon.c.tgz 2023-06-23T01:04:52.145 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2023-06-23T01:04:52.197 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.VeNIYE95fM 2023-06-23T01:04:52.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.VeNIYE95fM 2023-06-23T01:04:52.303 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.VeNIYE95fM 2023-06-23T01:04:52.387 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.VeNIYE95fM is 271KB 2023-06-23T01:04:52.481 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.VeNIYE95fM 2023-06-23T01:04:52.496 INFO:tasks.ceph:Cleaning ceph cluster... 2023-06-23T01:04:52.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-23T01:04:52.499 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-23T01:04:52.560 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-06-23T01:04:52.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-23T01:04:52.569 DEBUG:teuthology.orchestra.run.smithi152:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-23T01:04:52.596 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-23T01:04:52.624 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-23T01:04:52.631 INFO:tasks.ceph:Archiving crash dumps... 2023-06-23T01:04:52.633 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/remote/smithi005/crash 2023-06-23T01:04:52.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-23T01:04:52.671 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/remote/smithi152/crash 2023-06-23T01:04:52.673 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-23T01:04:52.697 INFO:tasks.ceph:Shutting down logrotate 2023-06-23T01:04:52.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T01:04:52.719 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-23T01:04:52.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-23T01:04:52.826 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-23T01:04:52.851 INFO:tasks.ceph:Compressing logs... 2023-06-23T01:04:52.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:04:52.869 DEBUG:teuthology.orchestra.run.smithi152:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:04:53.158 INFO:tasks.ceph:Archiving logs... 2023-06-23T01:04:53.159 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/remote/smithi005/log 2023-06-23T01:04:53.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T01:04:53.237 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/remote/smithi152/log 2023-06-23T01:04:53.238 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T01:04:53.299 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 1893, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi005 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-23T01:04:53.388 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5f86246513fd4bad8bcdda35c0e9003d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 1893, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 251, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph.py", line 450, in cephfs_setup fs = Filesystem(ctx, fs_config=temp, name=name, create=True) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 514, in __init__ self.create() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 717, in create self.set_allow_standby_replay(standby_replay) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 589, in set_allow_standby_replay self.set_var("allow_standby_replay", yes) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/cephfs/filesystem.py", line 574, in set_var self.mon_manager.raw_cluster_cmd("fs", "set", self.name, var, *a) File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_ceph_ceph-c_42ddc46e6e6582d8e29a5a982a81886fe028be2c/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi005 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false' 2023-06-23T01:04:53.394 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-06-23T01:04:53.407 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-23T01:04:53.429 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-23T01:04:53.429 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T01:04:53.447 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T01:04:53.535 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-23T01:04:53.545 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-23T01:04:53.546 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:04:53.549 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T01:04:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:bash: ntpq: command not found 2023-06-23T01:04:53.563 INFO:teuthology.orchestra.run.smithi152.stderr:bash: ntpq: command not found 2023-06-23T01:04:53.564 INFO:teuthology.orchestra.run.smithi005.stdout:210 Number of sources = 4 2023-06-23T01:04:53.564 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:04:53.565 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2023-06-23T01:04:53.565 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv01.front.sepia.ceph.com 2 6 177 0 +2114us[+2114us] +/- 82ms 2023-06-23T01:04:53.565 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv02.front.sepia.ceph.com 3 6 177 2 -1885us[-1759us] +/- 75ms 2023-06-23T01:04:53.565 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 2 +9425ns[ +135us] +/- 81ms 2023-06-23T01:04:53.565 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:04:53.566 INFO:teuthology.orchestra.run.smithi152.stdout:210 Number of sources = 4 2023-06-23T01:04:53.567 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T01:04:53.567 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2023-06-23T01:04:53.567 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 2 6 177 1 +2128us[+2120us] +/- 82ms 2023-06-23T01:04:53.567 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 177 1 -1879us[-1887us] +/- 75ms 2023-06-23T01:04:53.568 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 177 0 +17us[ +17us] +/- 81ms 2023-06-23T01:04:53.568 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T01:04:53.568 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-23T01:04:53.580 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-23T01:04:53.582 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-23T01:04:53.604 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T01:04:53.696 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T01:04:53.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T01:04:53.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:04:53.816 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T01:04:53.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:04:53.846 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-23T01:04:53.865 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-23T01:04:53.874 INFO:teuthology.task.internal:Duration was 709.484558 seconds 2023-06-23T01:04:53.874 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-23T01:04:53.883 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-23T01:04:53.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T01:04:53.886 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T01:04:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T01:04:53.931 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T01:04:54.113 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-23T01:04:54.113 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2023-06-23T01:04:54.114 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T01:04:54.173 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2023-06-23T01:04:54.174 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T01:04:54.193 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-23T01:04:54.194 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:04:54.216 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T01:04:54.270 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-23T01:04:54.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T01:04:54.293 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T01:04:54.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-23T01:04:54.390 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-23T01:04:54.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T01:04:54.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T01:04:54.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-23T01:04:54.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:04:54.459 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:04:54.482 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2023-06-23T01:04:54.505 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2023-06-23T01:04:54.519 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:04:54.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:04:54.558 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T01:04:54.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T01:04:54.575 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-23T01:04:54.593 INFO:teuthology.task.internal:Transferring archived files... 2023-06-23T01:04:54.594 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/remote/smithi005 2023-06-23T01:04:54.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T01:04:54.630 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927/remote/smithi152 2023-06-23T01:04:54.630 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T01:04:54.676 INFO:teuthology.task.internal:Removing archive directory... 2023-06-23T01:04:54.676 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T01:04:54.680 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T01:04:54.712 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-23T01:04:54.724 INFO:teuthology.task.internal:Not uploading archives. 2023-06-23T01:04:54.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-23T01:04:54.734 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-23T01:04:54.735 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T01:04:54.737 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T01:04:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 01:04 /home/ubuntu/cephtest 2023-06-23T01:04:54.766 INFO:teuthology.orchestra.run.smithi152.stdout: 262176 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 01:04 /home/ubuntu/cephtest 2023-06-23T01:04:54.768 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-23T01:04:54.778 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-23T01:04:54.800 INFO:teuthology.nuke:Checking targets against current locks 2023-06-23T01:04:54.830 DEBUG:teuthology.nuke:shortname: smithi005 2023-06-23T01:04:54.831 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T01:04:54.859 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:20.279799', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOk9iHkStIKdX0e2hhQMYKWsnjcyeUlJFQaprlN9vcrlgJf1zAQW5lMrINiyEFNp3vlF1/GrAcyz+sP9MvS+cw/8LR0T9x0QvVQcfLoXOkTDBQ0cSA0C8V0980X82UkdwUSLQ7Rc4sUFVAUIArN7JKZlBlMljAhMc5Apt1JwYYScC7akB13fw5PgmoG3MalaMfjasoUy2Cxmxew5tWs/8EHRqE1MDShXdgHOpanXR/hvx8JIRhVTOWkfb9X9aHNO1eLiedcd5nQ6Prp3CzU2O3ghJ/W4CGJ7fmB7YSW5b6b0nkgwuyjocXCCva5RIyYNlhk7FvKxMRe5oVZx7VtNG+SzAiSgqY9RjBgy3GK3rnkZXmi+UfAQuFGYXp9l5kzGe3L/NMkm7VC9QpV7ZNU5U/fAvEqVKw3fImJo9wzkoEp1s+PA6e8VVPB7nDRcwB8Dy5XsMImCa4Be17I67BoUCym7fZk7drk9vi0NDk4ELyGcPQmWGgTdGckES8KbgtRys='} 2023-06-23T01:04:54.863 DEBUG:teuthology.nuke:shortname: smithi152 2023-06-23T01:04:54.863 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T01:04:54.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311927', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 00:45:20.278191', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHVIx9CZ40xo/rF3tqoJjc/54NIwjw2o4uiIp8TEbajXxRciQF9qUi141lvr2Mf8sIm7p605GD9L8u7Mq/N2iSE='} 2023-06-23T01:04:54.927 INFO:teuthology.orchestra.console:Power off smithi005 2023-06-23T01:04:54.929 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T01:04:54.956 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:04:54.976 INFO:teuthology.orchestra.console:Power off smithi152 2023-06-23T01:04:54.977 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T01:04:55.004 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:04:58.990 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:04:59.030 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:05:03.106 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:05:03.144 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:05:03.320 INFO:teuthology.orchestra.console:Power off for smithi005 completed 2023-06-23T01:05:07.260 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T01:05:07.473 INFO:teuthology.orchestra.console:Power off for smithi152 completed 2023-06-23T01:05:07.578 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 709.4845576286316 failure_reason: 'Command failed on smithi005 with status 22: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5f86246513fd4bad8bcdda35c0e9003d status: fail success: false 2023-06-23T01:05:07.579 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T01:05:07.662 INFO:teuthology.run:FAIL