2023-01-25T02:35:46.062 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T02:35:46.063 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136602 --owner scheduled_yuriw@teuthology --description rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits} --name yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136602/orig.config.yaml 2023-01-25T02:35:46.098 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T02:35:46.173 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136602 branch: wip-yuri-testing-2023-01-23-0926 description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits} email: yweinstein@radhat.com first_in_suite: false job_id: '7136602' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri-testing-2023-01-23-0926 ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos service trim min: 5 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd objectstore: filestore osd op queue: debug_random osd op queue cut off: debug_random osd sloppy crc: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(TOO_FEW_PGS\) - but it is still running - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(AUTH_BAD_CAPS\) sha1: b0d2592529629ab4f86fed5991e556a87dcda563 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: flavor: default sha1: b0d2592529629ab4f86fed5991e556a87dcda563 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-yuri-testing-2023-01-23-0926 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri-testing-2023-01-23-0926 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b0d2592529629ab4f86fed5991e556a87dcda563 targets: smithi019.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWKmrrkTJTYAqtSJfveg6gKmkIyR25CC3odKcn/Pnr9ILns6nNCPhgUo/FRjOMuOh17ZjFBO+HciyrIE5YheYsDvZc7IFjlKGcyKGUwdsxg2xmgWSqPEfcqLjaUjcJfir5GS9Y5PoKwkWAOBQULL1zBQePfn8N7TC771Utwg5bup8KSzaFefUiGemUhLPr1VVURzHXx3Ecjqowoz3bp8yRnqi1NpvP7d5kyYxupBVyp8Rbg8IIqzx0n5+wYdLPw/kSdM7vsCywpbkkrkMBWa4t2ImvQnIuhFlBX0yRqdNYkeynGSNhdwoUII2a02LUIC+QDS4Q1MHHqrOUg1hsILKbJzdhaxJ9jstAOzZjvioNDWaiXVWEpIAJRcFrbObPIUty2LCMNpzGlrkp1MT4L7R6/XvBrTCPW8tc9pLsGPn65szhBVZg3SZ3Mx6hO4Evvq9iSyRDVvjyeFeKYp9n5NW1oxDGvNZsN3qXTeK7fhdJTjvaBe8o7YNmu6D0e8Rm1M= smithi101.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0sgtEPttGl4btdP1NXT+jebLPumtSqYs6wL2/vMovcojm5YIkaKGFQptgjglOFfEQwxzzVqdU3TlsJnaPkOo7+0c8diSvmmwAKFUp1mHgXk8hyRzIoDzA9yqwlulI4bL0Ekm0DZKqd5vgi9iI4l7xFTGfOOggQu8BOqHpvyBQ+c1vHm4bkXeI6Ke2ZQyud60g0ewuX/gIrLzNkwQNbQ+pEZYoSU/4k9RjmwfE3utfWyDmXkoR0Vhp/vw1GUakB6/cYyub1OOSXeus5wHuUT6g8xVFhb7nR3sVZqGXFXemhsYJ5u/AMhZqQD0c0RqOy+2HzSYCBWaGnfB/Lth5uRZxcuKGp8ZixCmKP89w/ecIc0kb9vbYydNiqRVgaK1dHFp2TyAXlsOg0kf12IQsMw2Hj/2++7RC2IaOUoHJF/KtnkzEM68BuUr1VHBYd9fdAa5yTQzrklBWCPKUT/UxxaM2FhOuVlCB9/tqP7HIHS5qr6pRUphdvn4KVxNUo4Op28= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 store_thrash: true thrash_delay: 1 thrash_many: true - workunit: clients: client.0: - mon/pool_ops.sh - mon/crush_ops.sh - mon/osd.sh - mon/caps.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:20:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T02:35:46.174 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa; will attempt to use it 2023-01-25T02:35:46.175 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa/tasks 2023-01-25T02:35:46.175 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T02:35:46.177 INFO:teuthology.task.internal:Checking packages... 2023-01-25T02:35:46.203 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b0d2592529629ab4f86fed5991e556a87dcda563' 2023-01-25T02:35:46.203 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T02:35:46.204 INFO:teuthology.packaging:ref: None 2023-01-25T02:35:46.204 INFO:teuthology.packaging:tag: None 2023-01-25T02:35:46.204 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-01-23-0926 2023-01-25T02:35:46.205 INFO:teuthology.packaging:sha1: b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T02:35:46.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-01-23-0926 2023-01-25T02:35:46.355 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1959.gb0d25925 2023-01-25T02:35:46.356 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T02:35:46.363 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T02:35:46.363 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T02:35:46.381 INFO:teuthology.task.internal:Saving configuration 2023-01-25T02:35:46.394 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T02:35:46.400 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T02:35:46.422 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 02:30:41.605198', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVmlPc2uYvUCZ21a6o2pgZ4OSMB3U/9vPYmyq+a19Hoio2C/3ND4Xqv6ud/sViLkEtwhpQt6ID4Wnx/HwFHQuE='} 2023-01-25T02:35:46.443 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136602', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 02:30:41.603788', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAfEa20kr6MA3XZtZz5z+aI1Bov6hTnQrUnByq0dFMwv3FwTZEi68F5UP47dch+UUBI06iOrfiQ4+rQ94HXDjP8='} 2023-01-25T02:35:46.443 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T02:35:46.449 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2023-01-25T02:35:46.449 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2023-01-25T02:35:46.450 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T02:35:46.513 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0aa0cd05e0>, signals=[15]) 2023-01-25T02:35:46.514 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T02:35:46.530 INFO:teuthology.task.internal:Opening connections... 2023-01-25T02:35:46.531 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2023-01-25T02:35:46.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T02:35:46.605 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi101.front.sepia.ceph.com 2023-01-25T02:35:46.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T02:35:46.672 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T02:35:46.679 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2023-01-25T02:35:46.703 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2023-01-25T02:35:46.704 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2023-01-25T02:35:46.765 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2023-01-25T02:35:46.765 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="8" 2023-01-25T02:35:46.766 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2023-01-25T02:35:46.766 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2023-01-25T02:35:46.766 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="8" 2023-01-25T02:35:46.766 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el8" 2023-01-25T02:35:46.766 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T02:35:46.767 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2023-01-25T02:35:46.767 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T02:35:46.767 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2023-01-25T02:35:46.767 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T02:35:46.767 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T02:35:46.768 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T02:35:46.768 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2023-01-25T02:35:46.790 DEBUG:teuthology.orchestra.run.smithi101:> uname -m 2023-01-25T02:35:46.810 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2023-01-25T02:35:46.810 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-01-25T02:35:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="CentOS Stream" 2023-01-25T02:35:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8" 2023-01-25T02:35:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:ID="centos" 2023-01-25T02:35:46.869 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="rhel fedora" 2023-01-25T02:35:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8" 2023-01-25T02:35:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8" 2023-01-25T02:35:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T02:35:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-01-25T02:35:46.870 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T02:35:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://centos.org/" 2023-01-25T02:35:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T02:35:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T02:35:46.871 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T02:35:46.872 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-01-25T02:35:46.889 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T02:35:46.899 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T02:35:46.912 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T02:35:46.913 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2023-01-25T02:35:46.915 DEBUG:teuthology.orchestra.run.smithi101:> test '!' -e /home/ubuntu/cephtest 2023-01-25T02:35:46.934 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T02:35:46.949 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T02:35:46.950 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2023-01-25T02:35:46.976 DEBUG:teuthology.orchestra.run.smithi101:> test -z $(ls -A /var/lib/ceph) 2023-01-25T02:35:47.019 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T02:35:47.069 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T02:35:47.082 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T02:35:47.083 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.f': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T02:35:47.083 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T02:35:47.084 DEBUG:teuthology.orchestra.run.smithi101:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T02:35:47.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T02:35:47.102 DEBUG:teuthology.orchestra.run.smithi101:> uname -r 2023-01-25T02:35:47.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T02:35:47.104 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2023-01-25T02:35:47.160 INFO:teuthology.orchestra.run.smithi101.stdout:4.18.0-408.el8.x86_64 2023-01-25T02:35:47.160 INFO:teuthology.task.kernel:Running kernel on smithi101: 4.18.0-408.el8.x86_64 2023-01-25T02:35:47.160 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum install -y kernel 2023-01-25T02:35:47.164 INFO:teuthology.orchestra.run.smithi019.stdout:4.18.0-408.el8.x86_64 2023-01-25T02:35:47.165 INFO:teuthology.task.kernel:Running kernel on smithi019: 4.18.0-408.el8.x86_64 2023-01-25T02:35:47.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2023-01-25T02:35:50.773 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-8 - AppStream 34 MB/s | 27 MB 00:00 2023-01-25T02:35:51.002 INFO:teuthology.orchestra.run.smithi101.stdout:CentOS-8 - AppStream 30 MB/s | 27 MB 00:00 2023-01-25T02:35:56.966 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-8 - BaseOS 31 MB/s | 27 MB 00:00 2023-01-25T02:35:57.052 INFO:teuthology.orchestra.run.smithi101.stdout:CentOS-8 - BaseOS 25 MB/s | 27 MB 00:01 2023-01-25T02:36:01.360 INFO:teuthology.orchestra.run.smithi101.stdout:CentOS-8 - Extras 111 kB/s | 18 kB 00:00 2023-01-25T02:36:01.473 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-8 - Extras 181 kB/s | 18 kB 00:00 2023-01-25T02:36:01.762 INFO:teuthology.orchestra.run.smithi101.stdout:CentOS-8 - PowerTools 21 MB/s | 5.6 MB 00:00 2023-01-25T02:36:01.908 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-8 - PowerTools 19 MB/s | 5.6 MB 00:00 2023-01-25T02:36:03.203 INFO:teuthology.orchestra.run.smithi101.stdout:Copr repo for python3-asyncssh owned by ceph 9.9 kB/s | 3.5 kB 00:00 2023-01-25T02:36:03.207 INFO:teuthology.orchestra.run.smithi019.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-01-25T02:36:04.553 INFO:teuthology.orchestra.run.smithi101.stdout:Extra Packages for Enterprise Linux 11 MB/s | 13 MB 00:01 2023-01-25T02:36:04.900 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 8.6 MB/s | 13 MB 00:01 2023-01-25T02:36:07.878 INFO:teuthology.orchestra.run.smithi101.stdout:lab-extras 416 kB/s | 24 kB 00:00 2023-01-25T02:36:08.433 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 490 kB/s | 24 kB 00:00 2023-01-25T02:36:11.026 INFO:teuthology.orchestra.run.smithi101.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T02:36:11.027 INFO:teuthology.orchestra.run.smithi101.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T02:36:11.196 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:36:11.196 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:36:11.196 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:36:11.197 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:36:11.197 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:36:11.197 INFO:teuthology.orchestra.run.smithi101.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T02:36:11.197 INFO:teuthology.orchestra.run.smithi101.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T02:36:11.197 INFO:teuthology.orchestra.run.smithi101.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T02:36:11.197 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:36:11.198 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:36:11.198 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:36:11.198 INFO:teuthology.orchestra.run.smithi101.stdout:Install 3 Packages 2023-01-25T02:36:11.198 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:36:11.198 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 85 M 2023-01-25T02:36:11.199 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 94 M 2023-01-25T02:36:11.199 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:36:11.907 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T02:36:11.908 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T02:36:12.073 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:36:12.074 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:36:12.074 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:36:12.074 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:36:12.075 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:36:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T02:36:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T02:36:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T02:36:12.075 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:36:12.076 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:36:12.076 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:36:12.076 INFO:teuthology.orchestra.run.smithi019.stdout:Install 3 Packages 2023-01-25T02:36:12.076 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:36:12.077 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 85 M 2023-01-25T02:36:12.077 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 94 M 2023-01-25T02:36:12.077 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:36:12.255 INFO:teuthology.orchestra.run.smithi101.stdout:(1/3): kernel-core-4.18.0-448.el8.x86_64.rpm 39 MB/s | 41 MB 00:01 2023-01-25T02:36:12.355 INFO:teuthology.orchestra.run.smithi101.stdout:(2/3): kernel-4.18.0-448.el8.x86_64.rpm 7.9 MB/s | 9.2 MB 00:01 2023-01-25T02:36:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 12 MB/s | 9.2 MB 00:00 2023-01-25T02:36:13.285 INFO:teuthology.orchestra.run.smithi019.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 34 MB/s | 41 MB 00:01 2023-01-25T02:36:13.297 INFO:teuthology.orchestra.run.smithi101.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 16 MB/s | 34 MB 00:02 2023-01-25T02:36:13.300 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:36:13.300 INFO:teuthology.orchestra.run.smithi101.stdout:Total 40 MB/s | 85 MB 00:02 2023-01-25T02:36:13.810 INFO:teuthology.orchestra.run.smithi019.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 20 MB/s | 34 MB 00:01 2023-01-25T02:36:13.812 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:36:13.813 INFO:teuthology.orchestra.run.smithi019.stdout:Total 49 MB/s | 85 MB 00:01 2023-01-25T02:36:13.933 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:36:14.301 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:36:14.301 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:36:14.476 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:36:14.894 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:36:14.895 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:36:16.384 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:36:16.411 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:36:16.865 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:36:16.905 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:36:19.099 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:36:19.674 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T02:36:19.811 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:36:20.493 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T02:36:23.427 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T02:36:23.848 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T02:36:24.417 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T02:36:24.817 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T02:36:27.165 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T02:36:27.605 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:28.369 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T02:36:28.654 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:49.957 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:49.983 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:50.390 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:50.391 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T02:36:50.391 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T02:36:50.975 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:50.976 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:36:50.976 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:36:50.976 INFO:teuthology.orchestra.run.smithi101.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T02:36:50.976 INFO:teuthology.orchestra.run.smithi101.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T02:36:50.977 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:36:50.977 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:36:51.096 DEBUG:teuthology.orchestra.run.smithi101:> rpm -q kernel --last | head -n 1 2023-01-25T02:36:51.290 INFO:teuthology.orchestra.run.smithi101.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 02:36:27 AM UTC 2023-01-25T02:36:51.292 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T02:36:51.292 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T02:36:51.292 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T02:36:51.293 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T02:36:51.293 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2023-01-25T02:36:51.293 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi101.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T02:36:51.293 DEBUG:teuthology.orchestra.run.smithi101:> rpm -q kernel --last | head -n 1 2023-01-25T02:36:51.480 INFO:teuthology.orchestra.run.smithi101.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 02:36:27 AM UTC 2023-01-25T02:36:51.481 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T02:36:51.482 DEBUG:teuthology.orchestra.run.smithi101:> sudo rpm -qi grub2-tools 2023-01-25T02:36:51.515 INFO:teuthology.orchestra.run.smithi101.stdout:Name : grub2-tools 2023-01-25T02:36:51.515 INFO:teuthology.orchestra.run.smithi101.stdout:Epoch : 1 2023-01-25T02:36:51.515 INFO:teuthology.orchestra.run.smithi101.stdout:Version : 2.02 2023-01-25T02:36:51.515 INFO:teuthology.orchestra.run.smithi101.stdout:Release : 129.el8 2023-01-25T02:36:51.516 INFO:teuthology.orchestra.run.smithi101.stdout:Architecture: x86_64 2023-01-25T02:36:51.516 INFO:teuthology.orchestra.run.smithi101.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T02:36:51.516 INFO:teuthology.orchestra.run.smithi101.stdout:Group : System Environment/Base 2023-01-25T02:36:51.516 INFO:teuthology.orchestra.run.smithi101.stdout:Size : 9265586 2023-01-25T02:36:51.517 INFO:teuthology.orchestra.run.smithi101.stdout:License : GPLv3+ 2023-01-25T02:36:51.517 INFO:teuthology.orchestra.run.smithi101.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T02:36:51.517 INFO:teuthology.orchestra.run.smithi101.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T02:36:51.517 INFO:teuthology.orchestra.run.smithi101.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T02:36:51.517 INFO:teuthology.orchestra.run.smithi101.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T02:36:51.517 INFO:teuthology.orchestra.run.smithi101.stdout:Relocations : (not relocatable) 2023-01-25T02:36:51.518 INFO:teuthology.orchestra.run.smithi101.stdout:Packager : CentOS BuildSystem 2023-01-25T02:36:51.518 INFO:teuthology.orchestra.run.smithi101.stdout:Vendor : CentOS 2023-01-25T02:36:51.518 INFO:teuthology.orchestra.run.smithi101.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T02:36:51.518 INFO:teuthology.orchestra.run.smithi101.stdout:Summary : Support tools for GRUB. 2023-01-25T02:36:51.518 INFO:teuthology.orchestra.run.smithi101.stdout:Description : 2023-01-25T02:36:51.519 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:36:51.519 INFO:teuthology.orchestra.run.smithi101.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T02:36:51.519 INFO:teuthology.orchestra.run.smithi101.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T02:36:51.519 INFO:teuthology.orchestra.run.smithi101.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T02:36:51.519 INFO:teuthology.orchestra.run.smithi101.stdout:hardware devices. 2023-01-25T02:36:51.520 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:36:51.520 INFO:teuthology.orchestra.run.smithi101.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T02:36:51.521 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T02:36:51.521 INFO:teuthology.task.kernel:Updating grub on smithi101 to boot 4.18.0-448.el8.x86_64 2023-01-25T02:36:51.522 DEBUG:teuthology.orchestra.run.smithi101:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T02:36:52.024 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:52.051 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:52.869 INFO:teuthology.orchestra.run.smithi101.stderr:Generating grub configuration file ... 2023-01-25T02:36:52.905 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:52.905 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T02:36:52.905 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T02:36:53.505 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T02:36:53.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:36:53.505 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:36:53.505 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T02:36:53.506 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T02:36:53.506 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:36:53.506 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:36:53.621 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel --last | head -n 1 2023-01-25T02:36:53.826 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 02:36:28 AM UTC 2023-01-25T02:36:53.828 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T02:36:53.828 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T02:36:53.828 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T02:36:53.828 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T02:36:53.829 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-01-25T02:36:53.829 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T02:36:53.829 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel --last | head -n 1 2023-01-25T02:36:54.025 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 02:36:28 AM UTC 2023-01-25T02:36:54.027 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T02:36:54.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2023-01-25T02:36:54.061 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2023-01-25T02:36:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2023-01-25T02:36:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.02 2023-01-25T02:36:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 129.el8 2023-01-25T02:36:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2023-01-25T02:36:54.062 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T02:36:54.063 INFO:teuthology.orchestra.run.smithi019.stdout:Group : System Environment/Base 2023-01-25T02:36:54.063 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 9265586 2023-01-25T02:36:54.063 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2023-01-25T02:36:54.063 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T02:36:54.064 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T02:36:54.064 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T02:36:54.064 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T02:36:54.064 INFO:teuthology.orchestra.run.smithi019.stdout:Relocations : (not relocatable) 2023-01-25T02:36:54.064 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : CentOS BuildSystem 2023-01-25T02:36:54.065 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2023-01-25T02:36:54.065 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T02:36:54.065 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2023-01-25T02:36:54.065 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2023-01-25T02:36:54.065 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:36:54.065 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T02:36:54.066 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T02:36:54.066 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T02:36:54.066 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2023-01-25T02:36:54.066 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:36:54.066 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T02:36:54.067 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T02:36:54.067 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 4.18.0-448.el8.x86_64 2023-01-25T02:36:54.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T02:36:54.931 INFO:teuthology.orchestra.run.smithi101.stderr:done 2023-01-25T02:36:54.933 DEBUG:teuthology.orchestra.run.smithi101:> mktemp 2023-01-25T02:36:54.961 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.8xN9ucMzqH 2023-01-25T02:36:54.961 DEBUG:teuthology.orchestra.run.smithi101:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8xN9ucMzqH 2023-01-25T02:36:55.075 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.8xN9ucMzqH 2023-01-25T02:36:55.188 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.8xN9ucMzqH is 6KB 2023-01-25T02:36:55.236 DEBUG:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.8xN9ucMzqH 2023-01-25T02:36:55.253 DEBUG:teuthology.orchestra.run.smithi101:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T02:36:55.325 INFO:teuthology.orchestra.run.smithi101.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T02:36:55.325 INFO:teuthology.orchestra.run.smithi101.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T02:36:55.325 INFO:teuthology.orchestra.run.smithi101.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T02:36:55.325 INFO:teuthology.orchestra.run.smithi101.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T02:36:55.326 DEBUG:teuthology.orchestra.run.smithi101:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T02:36:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2023-01-25T02:36:55.484 DEBUG:teuthology.orchestra.run.smithi101:> sudo shutdown -r now 2023-01-25T02:36:57.696 INFO:teuthology.orchestra.run.smithi019.stderr:done 2023-01-25T02:36:57.697 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2023-01-25T02:36:57.763 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.e1ncmIyISE 2023-01-25T02:36:57.764 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.e1ncmIyISE 2023-01-25T02:36:57.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.e1ncmIyISE 2023-01-25T02:36:57.942 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.e1ncmIyISE is 6KB 2023-01-25T02:36:57.990 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.e1ncmIyISE 2023-01-25T02:36:58.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T02:36:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T02:36:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T02:36:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T02:36:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T02:36:58.075 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T02:36:58.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2023-01-25T02:37:25.512 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2023-01-25T02:37:25.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-01-25T02:37:25.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T02:37:28.276 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-01-25T02:37:28.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-01-25T02:37:28.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T02:38:25.545 DEBUG:teuthology.orchestra.remote:timed out 2023-01-25T02:38:28.281 DEBUG:teuthology.orchestra.remote:timed out 2023-01-25T02:38:55.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-01-25T02:38:55.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T02:38:55.893 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-01-25T02:38:56.150 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-01-25T02:38:56.150 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "4.18.0-448.el8.x86_64"... 2023-01-25T02:38:56.151 DEBUG:teuthology.orchestra.run.smithi101:> uname -r 2023-01-25T02:38:56.210 INFO:teuthology.orchestra.run.smithi101.stdout:4.18.0-448.el8.x86_64 2023-01-25T02:38:56.210 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T02:38:56.210 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T02:38:56.210 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T02:38:57.211 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2023-01-25T02:38:57.212 DEBUG:teuthology.orchestra.run.smithi101:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T02:38:57.310 INFO:teuthology.orchestra.run.smithi101.stdout:ttyS1 2023-01-25T02:38:57.328 DEBUG:teuthology.parallel:result is None 2023-01-25T02:38:58.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-01-25T02:38:58.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T02:38:58.648 DEBUG:teuthology.orchestra.run.smithi019:> true 2023-01-25T02:38:58.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-01-25T02:38:58.927 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-01-25T02:38:58.927 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2023-01-25T02:38:58.987 INFO:teuthology.orchestra.run.smithi019.stdout:4.18.0-448.el8.x86_64 2023-01-25T02:38:58.987 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T02:38:58.987 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T02:38:58.988 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T02:38:59.989 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T02:38:59.989 DEBUG:teuthology.orchestra.run.smithi019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T02:39:00.086 INFO:teuthology.orchestra.run.smithi019.stdout:ttyS1 2023-01-25T02:39:00.103 DEBUG:teuthology.parallel:result is None 2023-01-25T02:39:00.103 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T02:39:00.111 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T02:39:00.112 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T02:39:00.114 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T02:39:00.135 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T02:39:00.142 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T02:39:00.151 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T02:39:00.151 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T02:39:00.176 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T02:39:00.214 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T02:39:00.220 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T02:39:00.221 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T02:39:00.245 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T02:39:00.285 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T02:39:00.288 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T02:39:00.297 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T02:39:00.300 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T02:39:00.301 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T02:39:00.309 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T02:39:00.309 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T02:39:00.341 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T02:39:00.372 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T02:39:00.381 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T02:39:00.381 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T02:39:00.413 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T02:39:00.431 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T02:39:00.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T02:39:00.569 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T02:39:00.661 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T02:39:00.701 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:39:00.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T02:39:00.770 DEBUG:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T02:39:00.821 DEBUG:teuthology.orchestra.run.smithi101:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T02:39:00.874 DEBUG:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T02:39:00.959 DEBUG:teuthology.orchestra.run.smithi101:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T02:39:01.003 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:39:01.004 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T02:39:01.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2023-01-25T02:39:01.073 DEBUG:teuthology.orchestra.run.smithi101:> sudo service rsyslog restart 2023-01-25T02:39:01.125 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T02:39:01.173 INFO:teuthology.orchestra.run.smithi101.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T02:39:01.541 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T02:39:01.549 INFO:teuthology.task.internal:Starting timer... 2023-01-25T02:39:01.549 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T02:39:01.563 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T02:39:01.574 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T02:39:01.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2023-01-25T02:39:01.639 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2023-01-25T02:39:01.640 DEBUG:teuthology.orchestra.run.smithi101:> sudo service auditd rotate 2023-01-25T02:39:01.687 INFO:teuthology.orchestra.run.smithi101.stdout:Rotating logs: 2023-01-25T02:39:01.689 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T02:39:01.689 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2023-01-25T02:39:01.711 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2023-01-25T02:39:01.711 DEBUG:teuthology.orchestra.run.smithi101:> /usr/sbin/getenforce 2023-01-25T02:39:01.740 INFO:teuthology.orchestra.run.smithi101.stdout:Permissive 2023-01-25T02:39:01.740 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi101.front.sepia.ceph.com': 'permissive'} 2023-01-25T02:39:01.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T02:39:01.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T02:39:01.780 DEBUG:teuthology.orchestra.run.smithi101:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T02:39:01.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T02:39:01.809 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T02:39:01.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2023-01-25T02:39:01.848 DEBUG:teuthology.orchestra.run.smithi101:> sudo /usr/sbin/setenforce permissive 2023-01-25T02:39:01.886 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T02:39:01.896 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-25T02:39:01.901 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-25T02:39:02.076 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T02:39:02.092 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T02:39:02.122 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi101.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T02:45:27.443 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi101.front.sepia.ceph.com')] 2023-01-25T02:45:27.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2023-01-25T02:45:27.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T02:45:27.513 DEBUG:teuthology.orchestra.run.smithi019:> true 2023-01-25T02:45:27.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-01-25T02:45:27.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T02:45:27.607 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-01-25T02:45:27.673 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T02:45:27.686 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T02:45:27.687 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T02:45:27.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T02:45:27.690 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T02:45:27.691 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T02:45:27.723 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T02:45:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T02:45:27.745 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T02:45:27.759 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T02:45:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2023-01-25T02:45:27.822 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2023-01-25T02:45:27.824 INFO:teuthology.orchestra.run.smithi101.stderr:sudo: ntpd: command not found 2023-01-25T02:45:27.835 INFO:teuthology.orchestra.run.smithi101.stdout:506 Cannot talk to daemon 2023-01-25T02:45:27.837 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T02:45:27.849 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T02:45:27.851 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T02:45:27.863 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T02:45:27.919 INFO:teuthology.orchestra.run.smithi019.stderr:bash: ntpq: command not found 2023-01-25T02:45:27.923 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T02:45:27.923 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2023-01-25T02:45:27.923 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T02:45:27.924 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T02:45:27.924 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T02:45:27.924 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T02:45:27.936 INFO:teuthology.orchestra.run.smithi101.stderr:bash: ntpq: command not found 2023-01-25T02:45:27.940 INFO:teuthology.orchestra.run.smithi101.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T02:45:27.940 INFO:teuthology.orchestra.run.smithi101.stdout:=============================================================================== 2023-01-25T02:45:27.940 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T02:45:27.941 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T02:45:27.941 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T02:45:27.941 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T02:45:27.942 INFO:teuthology.run_tasks:Running task install... 2023-01-25T02:45:27.950 DEBUG:teuthology.task.install:project ceph 2023-01-25T02:45:27.951 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}} 2023-01-25T02:45:27.951 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'} 2023-01-25T02:45:27.951 INFO:teuthology.task.install:Using flavor: default 2023-01-25T02:45:27.957 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T02:45:27.958 INFO:teuthology.task.install:extra packages: [] 2023-01-25T02:45:27.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo lsb_release -is 2023-01-25T02:45:28.003 INFO:teuthology.orchestra.run.smithi019.stdout:CentOSStream 2023-01-25T02:45:28.004 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T02:45:28.005 DEBUG:teuthology.orchestra.run.smithi101:> sudo lsb_release -is 2023-01-25T02:45:28.005 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T02:45:28.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo lsb_release -is 2023-01-25T02:45:28.044 INFO:teuthology.orchestra.run.smithi101.stdout:CentOSStream 2023-01-25T02:45:28.045 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T02:45:28.045 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T02:45:28.046 DEBUG:teuthology.orchestra.run.smithi101:> sudo lsb_release -is 2023-01-25T02:45:28.083 INFO:teuthology.orchestra.run.smithi019.stdout:CentOSStream 2023-01-25T02:45:28.085 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T02:45:28.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T02:45:28.124 INFO:teuthology.orchestra.run.smithi101.stdout:CentOSStream 2023-01-25T02:45:28.125 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T02:45:28.126 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T02:45:28.249 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T02:45:28.250 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T02:45:28.283 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T02:45:28.283 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T02:45:28.364 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T02:45:28.364 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:45:28.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T02:45:28.393 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T02:45:28.394 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:45:28.394 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T02:45:28.395 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T02:45:28.396 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T02:45:28.423 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T02:45:28.424 DEBUG:teuthology.orchestra.run.smithi101:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T02:45:28.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T02:45:28.492 DEBUG:teuthology.orchestra.run.smithi101:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T02:45:28.542 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T02:45:28.570 DEBUG:teuthology.orchestra.run.smithi101:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T02:45:28.609 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2023-01-25T02:45:28.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2023-01-25T02:45:28.637 INFO:teuthology.orchestra.run.smithi101.stdout:check_obsoletes = 1 2023-01-25T02:45:28.639 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum clean all 2023-01-25T02:45:28.871 INFO:teuthology.orchestra.run.smithi019.stdout:61 files removed 2023-01-25T02:45:28.894 INFO:teuthology.orchestra.run.smithi101.stdout:61 files removed 2023-01-25T02:45:28.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw 2023-01-25T02:45:28.922 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-radosgw 2023-01-25T02:45:29.777 INFO:teuthology.orchestra.run.smithi101.stdout:ceph packages for x86_64 178 kB/s | 80 kB 00:00 2023-01-25T02:45:29.869 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 147 kB/s | 80 kB 00:00 2023-01-25T02:45:30.061 INFO:teuthology.orchestra.run.smithi101.stdout:ceph noarch packages 38 kB/s | 9.3 kB 00:00 2023-01-25T02:45:30.184 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 34 kB/s | 9.3 kB 00:00 2023-01-25T02:45:30.337 INFO:teuthology.orchestra.run.smithi101.stdout:ceph source packages 6.0 kB/s | 1.5 kB 00:00 2023-01-25T02:45:30.467 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2023-01-25T02:45:31.422 INFO:teuthology.orchestra.run.smithi101.stdout:CentOS-8 - AppStream 26 MB/s | 27 MB 00:01 2023-01-25T02:45:31.587 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-8 - AppStream 25 MB/s | 27 MB 00:01 2023-01-25T02:45:37.289 INFO:teuthology.orchestra.run.smithi101.stdout:CentOS-8 - BaseOS 34 MB/s | 27 MB 00:00 2023-01-25T02:45:37.662 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-8 - BaseOS 36 MB/s | 27 MB 00:00 2023-01-25T02:45:41.572 INFO:teuthology.orchestra.run.smithi101.stdout:CentOS-8 - Extras 106 kB/s | 18 kB 00:00 2023-01-25T02:45:41.974 INFO:teuthology.orchestra.run.smithi101.stdout:CentOS-8 - PowerTools 21 MB/s | 5.6 MB 00:00 2023-01-25T02:45:42.191 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-8 - Extras 133 kB/s | 18 kB 00:00 2023-01-25T02:45:42.597 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-8 - PowerTools 21 MB/s | 5.6 MB 00:00 2023-01-25T02:45:43.198 INFO:teuthology.orchestra.run.smithi101.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-01-25T02:45:43.879 INFO:teuthology.orchestra.run.smithi019.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-01-25T02:45:44.307 INFO:teuthology.orchestra.run.smithi101.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-25T02:45:44.988 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-25T02:45:47.591 INFO:teuthology.orchestra.run.smithi101.stdout:lab-extras 434 kB/s | 24 kB 00:00 2023-01-25T02:45:48.447 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 380 kB/s | 24 kB 00:00 2023-01-25T02:45:51.021 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:45:51.023 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:45:51.024 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:45:51.024 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:45:51.024 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:45:51.024 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T02:45:51.025 INFO:teuthology.orchestra.run.smithi101.stdout:Upgrading: 2023-01-25T02:45:51.025 INFO:teuthology.orchestra.run.smithi101.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T02:45:51.025 INFO:teuthology.orchestra.run.smithi101.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T02:45:51.025 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-01-25T02:45:51.025 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T02:45:51.026 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T02:45:51.026 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T02:45:51.026 INFO:teuthology.orchestra.run.smithi101.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T02:45:51.026 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T02:45:51.027 INFO:teuthology.orchestra.run.smithi101.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T02:45:51.027 INFO:teuthology.orchestra.run.smithi101.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T02:45:51.027 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T02:45:51.027 INFO:teuthology.orchestra.run.smithi101.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T02:45:51.027 INFO:teuthology.orchestra.run.smithi101.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T02:45:51.028 INFO:teuthology.orchestra.run.smithi101.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T02:45:51.028 INFO:teuthology.orchestra.run.smithi101.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T02:45:51.028 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse 2023-01-25T02:45:51.028 INFO:teuthology.orchestra.run.smithi101.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T02:45:51.028 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T02:45:51.029 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T02:45:51.029 INFO:teuthology.orchestra.run.smithi101.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T02:45:51.029 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T02:45:51.029 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T02:45:51.029 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T02:45:51.029 INFO:teuthology.orchestra.run.smithi101.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T02:45:51.030 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:45:51.030 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:45:51.030 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:45:51.030 INFO:teuthology.orchestra.run.smithi101.stdout:Install 21 Packages 2023-01-25T02:45:51.030 INFO:teuthology.orchestra.run.smithi101.stdout:Upgrade 2 Packages 2023-01-25T02:45:51.031 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:45:51.031 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 62 M 2023-01-25T02:45:51.032 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:45:51.709 INFO:teuthology.orchestra.run.smithi101.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 7.6 MB/s | 5.2 MB 00:00 2023-01-25T02:45:51.759 INFO:teuthology.orchestra.run.smithi101.stdout:(2/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 487 kB/s | 24 kB 00:00 2023-01-25T02:45:51.851 INFO:teuthology.orchestra.run.smithi101.stdout:(3/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 8.2 MB/s | 768 kB 00:00 2023-01-25T02:45:51.943 INFO:teuthology.orchestra.run.smithi101.stdout:(4/23): libradosstriper1-18.0.0-1959.gb0d25925. 5.0 MB/s | 474 kB 00:00 2023-01-25T02:45:52.090 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:45:52.092 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:45:52.093 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:45:52.093 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:45:52.093 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:45:52.093 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T02:45:52.094 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2023-01-25T02:45:52.094 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T02:45:52.094 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T02:45:52.094 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-01-25T02:45:52.094 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T02:45:52.095 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T02:45:52.095 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T02:45:52.095 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T02:45:52.095 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T02:45:52.095 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T02:45:52.096 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T02:45:52.096 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T02:45:52.096 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T02:45:52.096 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T02:45:52.096 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T02:45:52.097 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T02:45:52.097 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse 2023-01-25T02:45:52.097 INFO:teuthology.orchestra.run.smithi019.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T02:45:52.097 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T02:45:52.097 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T02:45:52.098 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T02:45:52.098 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T02:45:52.098 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T02:45:52.098 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T02:45:52.099 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T02:45:52.099 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:45:52.099 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:45:52.099 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:45:52.099 INFO:teuthology.orchestra.run.smithi019.stdout:Install 21 Packages 2023-01-25T02:45:52.100 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 2 Packages 2023-01-25T02:45:52.100 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:45:52.101 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 62 M 2023-01-25T02:45:52.101 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:45:52.551 INFO:teuthology.orchestra.run.smithi101.stdout:(5/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_6 11 MB/s | 6.5 MB 00:00 2023-01-25T02:45:52.643 INFO:teuthology.orchestra.run.smithi101.stdout:(6/23): python3-ceph-argparse-18.0.0-1959.gb0d2 487 kB/s | 45 kB 00:00 2023-01-25T02:45:52.752 INFO:teuthology.orchestra.run.smithi101.stdout:(7/23): python3-ceph-common-18.0.0-1959.gb0d259 1.0 MB/s | 115 kB 00:00 2023-01-25T02:45:53.369 INFO:teuthology.orchestra.run.smithi101.stdout:(8/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 5.6 MB/s | 13 MB 00:02 2023-01-25T02:45:53.394 INFO:teuthology.orchestra.run.smithi101.stdout:(9/23): python3-cephfs-18.0.0-1959.gb0d25925.el 296 kB/s | 190 kB 00:00 2023-01-25T02:45:53.461 INFO:teuthology.orchestra.run.smithi101.stdout:(10/23): python3-rados-18.0.0-1959.gb0d25925.el 4.0 MB/s | 372 kB 00:00 2023-01-25T02:45:53.515 INFO:teuthology.orchestra.run.smithi019.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 3.7 MB/s | 5.2 MB 00:01 2023-01-25T02:45:53.536 INFO:teuthology.orchestra.run.smithi101.stdout:(11/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 2.5 MB/s | 364 kB 00:00 2023-01-25T02:45:53.562 INFO:teuthology.orchestra.run.smithi101.stdout:(12/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 1.0 MB/s | 107 kB 00:00 2023-01-25T02:45:53.565 INFO:teuthology.orchestra.run.smithi019.stdout:(2/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 488 kB/s | 24 kB 00:00 2023-01-25T02:45:53.637 INFO:teuthology.orchestra.run.smithi101.stdout:(13/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.5 MB/s | 354 kB 00:00 2023-01-25T02:45:53.723 INFO:teuthology.orchestra.run.smithi019.stdout:(3/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 4.7 MB/s | 768 kB 00:00 2023-01-25T02:45:53.824 INFO:teuthology.orchestra.run.smithi019.stdout:(4/23): libradosstriper1-18.0.0-1959.gb0d25925. 4.6 MB/s | 474 kB 00:00 2023-01-25T02:45:53.862 INFO:teuthology.orchestra.run.smithi101.stdout:(14/23): ceph-common-18.0.0-1959.gb0d25925.el8. 8.6 MB/s | 24 MB 00:02 2023-01-25T02:45:53.888 INFO:teuthology.orchestra.run.smithi101.stdout:(15/23): python3-prettytable-0.7.2-14.el8.noarc 175 kB/s | 44 kB 00:00 2023-01-25T02:45:53.921 INFO:teuthology.orchestra.run.smithi101.stdout:(16/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 732 kB/s | 259 kB 00:00 2023-01-25T02:45:53.946 INFO:teuthology.orchestra.run.smithi101.stdout:(17/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 558 kB/s | 47 kB 00:00 2023-01-25T02:45:54.055 INFO:teuthology.orchestra.run.smithi101.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 446 kB/s | 59 kB 00:00 2023-01-25T02:45:54.080 INFO:teuthology.orchestra.run.smithi101.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 561 kB/s | 75 kB 00:00 2023-01-25T02:45:54.130 INFO:teuthology.orchestra.run.smithi101.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-01-25T02:45:54.314 INFO:teuthology.orchestra.run.smithi101.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 6.7 MB/s | 1.7 MB 00:00 2023-01-25T02:45:54.606 INFO:teuthology.orchestra.run.smithi101.stdout:(22/23): librados2-18.0.0-1959.gb0d25925.el8.x8 6.9 MB/s | 3.6 MB 00:00 2023-01-25T02:45:54.616 INFO:teuthology.orchestra.run.smithi019.stdout:(5/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 5.2 MB/s | 13 MB 00:02 2023-01-25T02:45:54.674 INFO:teuthology.orchestra.run.smithi019.stdout:(6/23): python3-ceph-argparse-18.0.0-1959.gb0d2 768 kB/s | 45 kB 00:00 2023-01-25T02:45:54.689 INFO:teuthology.orchestra.run.smithi101.stdout:(23/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 6.4 MB/s | 3.6 MB 00:00 2023-01-25T02:45:54.693 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:45:54.693 INFO:teuthology.orchestra.run.smithi101.stdout:Total 17 MB/s | 62 MB 00:03 2023-01-25T02:45:54.732 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:45:54.776 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:45:54.776 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:45:54.799 INFO:teuthology.orchestra.run.smithi019.stdout:(7/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_6 6.6 MB/s | 6.5 MB 00:00 2023-01-25T02:45:54.825 INFO:teuthology.orchestra.run.smithi019.stdout:(8/23): python3-ceph-common-18.0.0-1959.gb0d259 764 kB/s | 115 kB 00:00 2023-01-25T02:45:54.891 INFO:teuthology.orchestra.run.smithi019.stdout:(9/23): python3-cephfs-18.0.0-1959.gb0d25925.el 2.8 MB/s | 190 kB 00:00 2023-01-25T02:45:54.917 INFO:teuthology.orchestra.run.smithi019.stdout:(10/23): python3-rados-18.0.0-1959.gb0d25925.el 4.0 MB/s | 372 kB 00:00 2023-01-25T02:45:54.959 INFO:teuthology.orchestra.run.smithi019.stdout:(11/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 5.3 MB/s | 364 kB 00:00 2023-01-25T02:45:55.009 INFO:teuthology.orchestra.run.smithi019.stdout:(12/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 1.1 MB/s | 107 kB 00:00 2023-01-25T02:45:55.076 INFO:teuthology.orchestra.run.smithi019.stdout:(13/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-01-25T02:45:55.093 INFO:teuthology.orchestra.run.smithi019.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 2.6 MB/s | 44 kB 00:00 2023-01-25T02:45:55.118 INFO:teuthology.orchestra.run.smithi019.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.4 MB/s | 259 kB 00:00 2023-01-25T02:45:55.135 INFO:teuthology.orchestra.run.smithi019.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-01-25T02:45:55.261 INFO:teuthology.orchestra.run.smithi019.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 470 kB/s | 59 kB 00:00 2023-01-25T02:45:55.327 INFO:teuthology.orchestra.run.smithi019.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-01-25T02:45:55.353 INFO:teuthology.orchestra.run.smithi019.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 818 kB/s | 75 kB 00:00 2023-01-25T02:45:55.418 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:45:55.422 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:45:55.728 INFO:teuthology.orchestra.run.smithi019.stdout:(20/23): ceph-common-18.0.0-1959.gb0d25925.el8. 6.7 MB/s | 24 MB 00:03 2023-01-25T02:45:55.762 INFO:teuthology.orchestra.run.smithi019.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 4.0 MB/s | 1.7 MB 00:00 2023-01-25T02:45:55.845 INFO:teuthology.orchestra.run.smithi019.stdout:(22/23): librados2-18.0.0-1959.gb0d25925.el8.x8 7.4 MB/s | 3.6 MB 00:00 2023-01-25T02:45:56.054 INFO:teuthology.orchestra.run.smithi019.stdout:(23/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 11 MB/s | 3.6 MB 00:00 2023-01-25T02:45:56.058 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:45:56.058 INFO:teuthology.orchestra.run.smithi019.stdout:Total 16 MB/s | 62 MB 00:03 2023-01-25T02:45:56.097 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:45:56.146 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:45:56.146 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:45:56.801 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:45:56.805 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:45:56.877 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:45:57.095 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T02:45:57.328 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T02:45:57.484 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T02:45:57.864 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T02:45:58.027 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:45:58.074 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T02:45:58.165 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T02:45:58.212 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T02:45:58.398 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T02:45:58.466 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T02:45:58.591 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T02:45:58.614 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T02:45:58.981 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T02:45:59.025 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T02:45:59.183 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T02:45:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T02:45:59.245 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T02:45:59.306 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T02:45:59.439 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T02:45:59.558 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T02:45:59.608 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T02:45:59.673 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T02:45:59.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T02:45:59.802 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T02:46:00.068 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T02:46:00.142 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T02:46:00.314 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T02:46:00.318 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T02:46:00.376 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T02:46:00.449 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T02:46:00.520 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T02:46:00.543 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T02:46:00.664 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T02:46:00.738 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T02:46:00.777 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T02:46:00.901 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T02:46:01.135 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T02:46:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T02:46:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T02:46:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T02:46:01.637 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T02:46:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T02:46:01.841 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 16/25 2023-01-25T02:46:01.909 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 16/25 2023-01-25T02:46:02.149 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T02:46:02.333 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T02:46:02.533 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T02:46:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T02:46:02.872 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 16/25 2023-01-25T02:46:02.949 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 16/25 2023-01-25T02:46:03.132 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T02:46:03.308 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T02:46:03.530 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T02:46:06.311 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T02:46:06.541 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T02:46:07.134 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T02:46:07.444 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T02:46:07.539 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T02:46:07.546 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T02:46:07.732 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T02:46:07.747 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T02:46:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T02:46:08.691 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T02:46:08.769 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T02:46:08.930 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T02:46:26.983 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T02:46:26.984 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /sys 2023-01-25T02:46:26.984 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /proc 2023-01-25T02:46:26.984 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /dev 2023-01-25T02:46:26.984 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /run 2023-01-25T02:46:26.985 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /mnt 2023-01-25T02:46:26.985 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /var/tmp 2023-01-25T02:46:26.985 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /home 2023-01-25T02:46:26.985 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /tmp 2023-01-25T02:46:26.985 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /dev 2023-01-25T02:46:26.985 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:28.424 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T02:46:28.445 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T02:46:28.445 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T02:46:28.649 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T02:46:28.649 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T02:46:28.834 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T02:46:28.835 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2023-01-25T02:46:28.835 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2023-01-25T02:46:28.835 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /dev 2023-01-25T02:46:28.835 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /run 2023-01-25T02:46:28.835 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2023-01-25T02:46:28.836 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2023-01-25T02:46:28.836 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2023-01-25T02:46:28.836 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2023-01-25T02:46:28.836 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /dev 2023-01-25T02:46:28.836 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:29.405 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T02:46:29.405 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T02:46:29.405 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T02:46:29.406 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T02:46:29.406 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T02:46:29.406 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T02:46:29.406 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T02:46:29.406 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T02:46:29.407 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T02:46:29.407 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T02:46:29.407 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T02:46:29.407 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T02:46:29.407 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T02:46:29.407 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T02:46:29.408 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T02:46:29.408 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T02:46:29.408 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T02:46:29.408 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T02:46:29.408 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T02:46:29.409 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T02:46:29.409 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T02:46:29.409 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T02:46:29.409 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T02:46:29.409 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T02:46:29.410 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T02:46:30.274 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T02:46:30.338 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:30.339 INFO:teuthology.orchestra.run.smithi101.stdout:Upgraded: 2023-01-25T02:46:30.339 INFO:teuthology.orchestra.run.smithi101.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.339 INFO:teuthology.orchestra.run.smithi101.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.339 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:46:30.340 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.340 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.340 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.340 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.341 INFO:teuthology.orchestra.run.smithi101.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T02:46:30.341 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.341 INFO:teuthology.orchestra.run.smithi101.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T02:46:30.341 INFO:teuthology.orchestra.run.smithi101.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T02:46:30.341 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.342 INFO:teuthology.orchestra.run.smithi101.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T02:46:30.342 INFO:teuthology.orchestra.run.smithi101.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.342 INFO:teuthology.orchestra.run.smithi101.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T02:46:30.342 INFO:teuthology.orchestra.run.smithi101.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T02:46:30.342 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.343 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.343 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.343 INFO:teuthology.orchestra.run.smithi101.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T02:46:30.343 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.343 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.344 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:30.344 INFO:teuthology.orchestra.run.smithi101.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T02:46:30.344 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:30.344 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:46:30.346 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T02:46:30.368 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T02:46:30.369 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T02:46:30.567 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-test 2023-01-25T02:46:30.571 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T02:46:30.571 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T02:46:31.282 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T02:46:31.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T02:46:31.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T02:46:31.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T02:46:31.283 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T02:46:31.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T02:46:31.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T02:46:31.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T02:46:31.284 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T02:46:31.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T02:46:31.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T02:46:31.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T02:46:31.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T02:46:31.285 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T02:46:31.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T02:46:31.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T02:46:31.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T02:46:31.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T02:46:31.286 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T02:46:31.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T02:46:31.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T02:46:31.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T02:46:31.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T02:46:31.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T02:46:31.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T02:46:31.375 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:00:44 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:46:32.154 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T02:46:32.154 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:32.155 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2023-01-25T02:46:32.155 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.155 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.155 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:46:32.155 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.156 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.156 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.156 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.156 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T02:46:32.156 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.157 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T02:46:32.157 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T02:46:32.157 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.157 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T02:46:32.157 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.157 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T02:46:32.158 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T02:46:32.158 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.158 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.158 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.158 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T02:46:32.159 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.159 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.159 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:32.159 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T02:46:32.160 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:32.160 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:46:32.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-test 2023-01-25T02:46:32.501 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:46:32.502 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:46:32.502 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:46:32.502 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:46:32.503 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:46:32.503 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T02:46:32.503 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-01-25T02:46:32.503 INFO:teuthology.orchestra.run.smithi101.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T02:46:32.503 INFO:teuthology.orchestra.run.smithi101.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T02:46:32.503 INFO:teuthology.orchestra.run.smithi101.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T02:46:32.504 INFO:teuthology.orchestra.run.smithi101.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T02:46:32.504 INFO:teuthology.orchestra.run.smithi101.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T02:46:32.504 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:32.504 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:46:32.505 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:46:32.505 INFO:teuthology.orchestra.run.smithi101.stdout:Install 6 Packages 2023-01-25T02:46:32.505 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:32.506 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 48 M 2023-01-25T02:46:32.506 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 218 M 2023-01-25T02:46:32.506 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:46:32.685 INFO:teuthology.orchestra.run.smithi101.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2023-01-25T02:46:32.727 INFO:teuthology.orchestra.run.smithi101.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-01-25T02:46:32.752 INFO:teuthology.orchestra.run.smithi101.stdout:(3/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 553 kB/s | 137 kB 00:00 2023-01-25T02:46:32.786 INFO:teuthology.orchestra.run.smithi101.stdout:(4/6): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-01-25T02:46:32.861 INFO:teuthology.orchestra.run.smithi101.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 651 kB/s | 71 kB 00:00 2023-01-25T02:46:33.245 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:45 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:46:34.486 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:46:34.487 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:46:34.488 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:46:34.488 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:46:34.488 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:46:34.488 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T02:46:34.488 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-01-25T02:46:34.489 INFO:teuthology.orchestra.run.smithi019.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T02:46:34.489 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T02:46:34.489 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T02:46:34.489 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T02:46:34.489 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T02:46:34.490 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:34.490 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:46:34.490 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:46:34.490 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2023-01-25T02:46:34.490 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:34.491 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 48 M 2023-01-25T02:46:34.491 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 218 M 2023-01-25T02:46:34.492 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:46:34.568 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.5 MB/s | 202 kB 00:00 2023-01-25T02:46:34.602 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 5.7 MB/s | 187 kB 00:00 2023-01-25T02:46:34.635 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.5 MB/s | 323 kB 00:00 2023-01-25T02:46:34.661 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 71 kB 00:00 2023-01-25T02:46:34.711 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 621 kB/s | 137 kB 00:00 2023-01-25T02:46:34.928 INFO:teuthology.orchestra.run.smithi101.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 20 MB/s | 47 MB 00:02 2023-01-25T02:46:34.932 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:46:34.932 INFO:teuthology.orchestra.run.smithi101.stdout:Total 20 MB/s | 48 MB 00:02 2023-01-25T02:46:34.976 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:46:34.997 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:46:34.997 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:46:35.293 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:46:35.297 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:46:35.784 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:46:35.999 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T02:46:36.231 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T02:46:36.428 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T02:46:36.508 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T02:46:36.698 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T02:46:36.861 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 20 MB/s | 47 MB 00:02 2023-01-25T02:46:36.863 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:46:36.864 INFO:teuthology.orchestra.run.smithi019.stdout:Total 20 MB/s | 48 MB 00:02 2023-01-25T02:46:36.886 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T02:46:36.906 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:46:36.928 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:46:36.929 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:46:37.240 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:46:37.243 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:46:37.746 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:46:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T02:46:38.180 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T02:46:38.334 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T02:46:38.416 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T02:46:38.588 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T02:46:38.775 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T02:46:41.513 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T02:46:41.808 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T02:46:42.114 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T02:46:42.114 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T02:46:42.115 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T02:46:42.115 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T02:46:42.115 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T02:46:42.115 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T02:46:42.678 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T02:46:42.678 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:42.678 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:46:42.679 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:42.679 INFO:teuthology.orchestra.run.smithi101.stdout: jq-1.6-6.el8.x86_64 2023-01-25T02:46:42.679 INFO:teuthology.orchestra.run.smithi101.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:42.679 INFO:teuthology.orchestra.run.smithi101.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T02:46:42.679 INFO:teuthology.orchestra.run.smithi101.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T02:46:42.680 INFO:teuthology.orchestra.run.smithi101.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T02:46:42.680 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:42.680 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:46:42.890 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph 2023-01-25T02:46:43.619 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:00:56 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:46:43.709 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T02:46:44.006 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T02:46:44.320 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T02:46:44.320 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T02:46:44.321 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T02:46:44.321 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T02:46:44.321 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T02:46:44.321 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T02:46:44.770 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:46:44.773 INFO:teuthology.orchestra.run.smithi101.stdout:======================================================================================= 2023-01-25T02:46:44.774 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:46:44.774 INFO:teuthology.orchestra.run.smithi101.stdout:======================================================================================= 2023-01-25T02:46:44.774 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:46:44.774 INFO:teuthology.orchestra.run.smithi101.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T02:46:44.774 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-01-25T02:46:44.775 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T02:46:44.775 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T02:46:44.775 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T02:46:44.775 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T02:46:44.775 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T02:46:44.775 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T02:46:44.776 INFO:teuthology.orchestra.run.smithi101.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T02:46:44.776 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T02:46:44.776 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T02:46:44.776 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T02:46:44.776 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T02:46:44.777 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T02:46:44.777 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T02:46:44.777 INFO:teuthology.orchestra.run.smithi101.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T02:46:44.777 INFO:teuthology.orchestra.run.smithi101.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T02:46:44.777 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T02:46:44.778 INFO:teuthology.orchestra.run.smithi101.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T02:46:44.778 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T02:46:44.778 INFO:teuthology.orchestra.run.smithi101.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T02:46:44.778 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T02:46:44.778 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T02:46:44.779 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T02:46:44.779 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T02:46:44.779 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T02:46:44.779 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T02:46:44.779 INFO:teuthology.orchestra.run.smithi101.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T02:46:44.779 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T02:46:44.780 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T02:46:44.780 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T02:46:44.780 INFO:teuthology.orchestra.run.smithi101.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T02:46:44.780 INFO:teuthology.orchestra.run.smithi101.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T02:46:44.780 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:44.781 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:46:44.781 INFO:teuthology.orchestra.run.smithi101.stdout:======================================================================================= 2023-01-25T02:46:44.781 INFO:teuthology.orchestra.run.smithi101.stdout:Install 32 Packages 2023-01-25T02:46:44.781 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:44.782 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 28 M 2023-01-25T02:46:44.782 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 110 M 2023-01-25T02:46:44.782 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:46:44.849 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T02:46:44.849 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:44.849 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:46:44.849 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:44.850 INFO:teuthology.orchestra.run.smithi019.stdout: jq-1.6-6.el8.x86_64 2023-01-25T02:46:44.850 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:44.850 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T02:46:44.850 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T02:46:44.850 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T02:46:44.851 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:44.851 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:46:44.913 INFO:teuthology.orchestra.run.smithi101.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 48 kB/s | 6.4 kB 00:00 2023-01-25T02:46:45.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph 2023-01-25T02:46:45.155 INFO:teuthology.orchestra.run.smithi101.stdout:(2/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 6.0 MB/s | 2.3 MB 00:00 2023-01-25T02:46:45.197 INFO:teuthology.orchestra.run.smithi101.stdout:(3/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 3.7 MB/s | 1.5 MB 00:00 2023-01-25T02:46:45.303 INFO:teuthology.orchestra.run.smithi101.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 2.2 MB/s | 236 kB 00:00 2023-01-25T02:46:45.364 INFO:teuthology.orchestra.run.smithi101.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.5 MB/s | 157 kB 00:00 2023-01-25T02:46:45.389 INFO:teuthology.orchestra.run.smithi101.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-01-25T02:46:45.415 INFO:teuthology.orchestra.run.smithi101.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.0 MB/s | 103 kB 00:00 2023-01-25T02:46:45.440 INFO:teuthology.orchestra.run.smithi101.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-01-25T02:46:45.474 INFO:teuthology.orchestra.run.smithi101.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2023-01-25T02:46:45.507 INFO:teuthology.orchestra.run.smithi101.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 7.0 MB/s | 237 kB 00:00 2023-01-25T02:46:45.541 INFO:teuthology.orchestra.run.smithi101.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-01-25T02:46:45.566 INFO:teuthology.orchestra.run.smithi101.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-01-25T02:46:45.708 INFO:teuthology.orchestra.run.smithi101.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 308 kB/s | 44 kB 00:00 2023-01-25T02:46:45.811 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:57 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:46:45.817 INFO:teuthology.orchestra.run.smithi101.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.7 MB/s | 185 kB 00:00 2023-01-25T02:46:45.934 INFO:teuthology.orchestra.run.smithi101.stdout:(15/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86 3.8 MB/s | 3.9 MB 00:01 2023-01-25T02:46:45.959 INFO:teuthology.orchestra.run.smithi101.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.2 MB/s | 173 kB 00:00 2023-01-25T02:46:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:(17/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 17 MB/s | 17 MB 00:00 2023-01-25T02:46:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 55 kB/s | 11 kB 00:00 2023-01-25T02:46:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 263 kB/s | 18 kB 00:00 2023-01-25T02:46:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.2 MB/s | 384 kB 00:00 2023-01-25T02:46:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 879 kB/s | 59 kB 00:00 2023-01-25T02:46:46.311 INFO:teuthology.orchestra.run.smithi101.stdout:(22/32): python3-logutils-0.3.5-11.el8.noarch.r 323 kB/s | 49 kB 00:00 2023-01-25T02:46:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 232 kB/s | 16 kB 00:00 2023-01-25T02:46:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-01-25T02:46:46.395 INFO:teuthology.orchestra.run.smithi101.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 222 kB/s | 19 kB 00:00 2023-01-25T02:46:46.412 INFO:teuthology.orchestra.run.smithi101.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 315 kB/s | 24 kB 00:00 2023-01-25T02:46:46.437 INFO:teuthology.orchestra.run.smithi101.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 437 kB/s | 29 kB 00:00 2023-01-25T02:46:46.455 INFO:teuthology.orchestra.run.smithi101.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 656 kB/s | 27 kB 00:00 2023-01-25T02:46:46.521 INFO:teuthology.orchestra.run.smithi101.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.2 MB/s | 241 kB 00:00 2023-01-25T02:46:46.547 INFO:teuthology.orchestra.run.smithi101.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.2 MB/s | 251 kB 00:00 2023-01-25T02:46:46.572 INFO:teuthology.orchestra.run.smithi101.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 730 kB/s | 85 kB 00:00 2023-01-25T02:46:46.588 INFO:teuthology.orchestra.run.smithi101.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 343 kB/s | 23 kB 00:00 2023-01-25T02:46:46.591 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:46:46.591 INFO:teuthology.orchestra.run.smithi101.stdout:Total 16 MB/s | 28 MB 00:01 2023-01-25T02:46:46.667 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:46:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:46:46.703 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:46:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:46:47.042 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-01-25T02:46:47.043 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:46:47.043 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-01-25T02:46:47.043 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:46:47.043 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T02:46:47.043 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-01-25T02:46:47.044 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T02:46:47.044 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T02:46:47.044 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T02:46:47.044 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T02:46:47.045 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T02:46:47.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T02:46:47.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T02:46:47.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T02:46:47.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T02:46:47.046 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T02:46:47.046 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T02:46:47.046 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T02:46:47.046 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T02:46:47.046 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T02:46:47.047 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T02:46:47.047 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T02:46:47.047 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T02:46:47.047 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T02:46:47.047 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T02:46:47.048 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T02:46:47.048 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T02:46:47.048 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T02:46:47.048 INFO:teuthology.orchestra.run.smithi019.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T02:46:47.048 INFO:teuthology.orchestra.run.smithi019.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T02:46:47.049 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T02:46:47.049 INFO:teuthology.orchestra.run.smithi019.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T02:46:47.049 INFO:teuthology.orchestra.run.smithi019.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T02:46:47.049 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T02:46:47.049 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T02:46:47.050 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T02:46:47.050 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T02:46:47.050 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:47.050 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:46:47.050 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2023-01-25T02:46:47.051 INFO:teuthology.orchestra.run.smithi019.stdout:Install 32 Packages 2023-01-25T02:46:47.051 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:46:47.051 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 28 M 2023-01-25T02:46:47.052 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 110 M 2023-01-25T02:46:47.052 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:46:47.076 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:46:47.079 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:46:47.188 INFO:teuthology.orchestra.run.smithi019.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 46 kB/s | 6.4 kB 00:00 2023-01-25T02:46:47.414 INFO:teuthology.orchestra.run.smithi019.stdout:(2/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 4.2 MB/s | 1.5 MB 00:00 2023-01-25T02:46:47.464 INFO:teuthology.orchestra.run.smithi019.stdout:(3/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 5.5 MB/s | 2.3 MB 00:00 2023-01-25T02:46:47.539 INFO:teuthology.orchestra.run.smithi019.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 3.1 MB/s | 236 kB 00:00 2023-01-25T02:46:47.598 INFO:teuthology.orchestra.run.smithi019.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T02:46:47.623 INFO:teuthology.orchestra.run.smithi019.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-01-25T02:46:47.648 INFO:teuthology.orchestra.run.smithi019.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.0 MB/s | 103 kB 00:00 2023-01-25T02:46:47.665 INFO:teuthology.orchestra.run.smithi019.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 3.2 MB/s | 54 kB 00:00 2023-01-25T02:46:47.730 INFO:teuthology.orchestra.run.smithi019.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 7.0 MB/s | 457 kB 00:00 2023-01-25T02:46:47.766 INFO:teuthology.orchestra.run.smithi019.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 6.4 MB/s | 237 kB 00:00 2023-01-25T02:46:47.850 INFO:teuthology.orchestra.run.smithi019.stdout:(11/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86 5.9 MB/s | 3.9 MB 00:00 2023-01-25T02:46:47.883 INFO:teuthology.orchestra.run.smithi019.stdout:(12/32): python3-cryptography-3.2.1-5.el8.x86_6 4.7 MB/s | 559 kB 00:00 2023-01-25T02:46:47.909 INFO:teuthology.orchestra.run.smithi019.stdout:(13/32): python3-pycparser-2.14-14.el8.noarch.r 1.8 MB/s | 109 kB 00:00 2023-01-25T02:46:48.018 INFO:teuthology.orchestra.run.smithi019.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 330 kB/s | 44 kB 00:00 2023-01-25T02:46:48.068 INFO:teuthology.orchestra.run.smithi019.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.1 MB/s | 185 kB 00:00 2023-01-25T02:46:48.127 INFO:teuthology.orchestra.run.smithi019.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.6 MB/s | 173 kB 00:00 2023-01-25T02:46:48.152 INFO:teuthology.orchestra.run.smithi019.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 4.5 MB/s | 384 kB 00:00 2023-01-25T02:46:48.177 INFO:teuthology.orchestra.run.smithi019.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-01-25T02:46:48.202 INFO:teuthology.orchestra.run.smithi019.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 350 kB/s | 18 kB 00:00 2023-01-25T02:46:48.324 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:46:48.403 INFO:teuthology.orchestra.run.smithi019.stdout:(20/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 17 MB/s | 17 MB 00:00 2023-01-25T02:46:48.428 INFO:teuthology.orchestra.run.smithi019.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 194 kB/s | 49 kB 00:00 2023-01-25T02:46:48.453 INFO:teuthology.orchestra.run.smithi019.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 234 kB/s | 59 kB 00:00 2023-01-25T02:46:48.503 INFO:teuthology.orchestra.run.smithi019.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 376 kB/s | 19 kB 00:00 2023-01-25T02:46:48.529 INFO:teuthology.orchestra.run.smithi019.stdout:(24/32): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2023-01-25T02:46:48.554 INFO:teuthology.orchestra.run.smithi019.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 475 kB/s | 24 kB 00:00 2023-01-25T02:46:48.579 INFO:teuthology.orchestra.run.smithi019.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 581 kB/s | 29 kB 00:00 2023-01-25T02:46:48.581 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T02:46:48.605 INFO:teuthology.orchestra.run.smithi019.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 547 kB/s | 27 kB 00:00 2023-01-25T02:46:48.630 INFO:teuthology.orchestra.run.smithi019.stdout:(28/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.2 MB/s | 283 kB 00:00 2023-01-25T02:46:48.664 INFO:teuthology.orchestra.run.smithi019.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.8 MB/s | 241 kB 00:00 2023-01-25T02:46:48.689 INFO:teuthology.orchestra.run.smithi019.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.4 MB/s | 85 kB 00:00 2023-01-25T02:46:48.714 INFO:teuthology.orchestra.run.smithi019.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.2 MB/s | 251 kB 00:00 2023-01-25T02:46:48.739 INFO:teuthology.orchestra.run.smithi019.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 305 kB/s | 23 kB 00:00 2023-01-25T02:46:48.743 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:46:48.743 INFO:teuthology.orchestra.run.smithi019.stdout:Total 17 MB/s | 28 MB 00:01 2023-01-25T02:46:48.825 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:46:48.842 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T02:46:48.866 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:46:48.866 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:46:49.072 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T02:46:49.257 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:46:49.260 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:46:49.308 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T02:46:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T02:46:49.694 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T02:46:49.896 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T02:46:50.065 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T02:46:50.290 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T02:46:50.477 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:46:50.552 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T02:46:50.717 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T02:46:50.789 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T02:46:50.974 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T02:46:51.043 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T02:46:51.209 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T02:46:51.424 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T02:46:51.472 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T02:46:51.670 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T02:46:51.704 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T02:46:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T02:46:51.909 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T02:46:52.018 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T02:46:52.149 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T02:46:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T02:46:52.376 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T02:46:52.434 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T02:46:52.677 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T02:46:52.702 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T02:46:52.915 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T02:46:52.949 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T02:46:53.136 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T02:46:53.188 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T02:46:53.339 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T02:46:53.557 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T02:46:53.564 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T02:46:53.783 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T02:46:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T02:46:53.957 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T02:46:54.011 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T02:46:54.226 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T02:46:54.354 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T02:46:54.441 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T02:46:54.706 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T02:46:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T02:46:54.980 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T02:46:55.069 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T02:46:55.168 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T02:46:55.280 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T02:46:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T02:46:55.627 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T02:46:55.861 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T02:46:56.137 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T02:46:56.485 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T02:46:56.776 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T02:46:56.885 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T02:46:56.981 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T02:46:57.245 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T02:46:57.367 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T02:46:57.431 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T02:46:57.584 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T02:46:57.797 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T02:46:57.959 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T02:46:57.976 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T02:46:58.115 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T02:46:58.545 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T02:46:58.545 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T02:46:58.545 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T02:46:58.546 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T02:46:58.546 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T02:46:58.546 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T02:46:58.546 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T02:46:58.546 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T02:46:58.547 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T02:46:58.547 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T02:46:58.547 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T02:46:58.547 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T02:46:58.547 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T02:46:58.548 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T02:46:58.548 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T02:46:58.548 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T02:46:58.548 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T02:46:58.549 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T02:46:58.549 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T02:46:58.549 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T02:46:58.549 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T02:46:58.549 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T02:46:58.549 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T02:46:58.550 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T02:46:58.550 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T02:46:58.550 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T02:46:58.550 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T02:46:58.550 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T02:46:58.551 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T02:46:58.551 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T02:46:58.552 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T02:46:58.552 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T02:46:59.020 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T02:46:59.262 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T02:46:59.512 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T02:46:59.512 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:59.513 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:46:59.513 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:59.513 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:59.513 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:59.513 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:46:59.514 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:59.514 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:46:59.514 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T02:46:59.514 INFO:teuthology.orchestra.run.smithi101.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T02:46:59.514 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T02:46:59.515 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T02:46:59.515 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T02:46:59.515 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T02:46:59.515 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T02:46:59.515 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T02:46:59.515 INFO:teuthology.orchestra.run.smithi101.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T02:46:59.516 INFO:teuthology.orchestra.run.smithi101.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T02:46:59.516 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T02:46:59.516 INFO:teuthology.orchestra.run.smithi101.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T02:46:59.516 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T02:46:59.516 INFO:teuthology.orchestra.run.smithi101.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T02:46:59.517 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T02:46:59.517 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T02:46:59.517 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T02:46:59.517 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T02:46:59.517 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T02:46:59.518 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T02:46:59.518 INFO:teuthology.orchestra.run.smithi101.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T02:46:59.518 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T02:46:59.518 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T02:46:59.518 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T02:46:59.519 INFO:teuthology.orchestra.run.smithi101.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T02:46:59.519 INFO:teuthology.orchestra.run.smithi101.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T02:46:59.519 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:46:59.519 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:46:59.678 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T02:46:59.722 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-base 2023-01-25T02:46:59.890 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T02:47:00.115 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T02:47:00.289 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T02:47:00.308 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T02:47:00.438 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T02:47:00.439 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:13 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:00.919 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T02:47:00.919 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T02:47:00.919 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T02:47:00.919 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T02:47:00.920 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T02:47:00.920 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T02:47:00.920 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T02:47:00.920 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T02:47:00.920 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T02:47:00.921 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T02:47:00.921 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T02:47:00.921 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T02:47:00.921 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T02:47:00.921 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T02:47:00.921 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T02:47:00.922 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T02:47:00.922 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T02:47:00.922 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T02:47:00.922 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T02:47:00.922 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T02:47:00.923 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T02:47:00.923 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T02:47:00.923 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T02:47:00.923 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T02:47:00.923 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T02:47:00.923 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T02:47:00.924 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T02:47:00.924 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T02:47:00.924 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T02:47:00.924 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T02:47:00.925 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T02:47:00.925 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T02:47:01.484 INFO:teuthology.orchestra.run.smithi101.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:47:01.582 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:01.584 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:47:01.584 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:01.676 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install cephadm 2023-01-25T02:47:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T02:47:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:01.946 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:47:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:47:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:47:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:47:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:47:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T02:47:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T02:47:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T02:47:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T02:47:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T02:47:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T02:47:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T02:47:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T02:47:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T02:47:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T02:47:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T02:47:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T02:47:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T02:47:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T02:47:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T02:47:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T02:47:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T02:47:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T02:47:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T02:47:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T02:47:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T02:47:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T02:47:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T02:47:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T02:47:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T02:47:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T02:47:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:01.953 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:02.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-base 2023-01-25T02:47:02.387 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:15 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:02.927 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:14 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:03.533 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:03.534 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:03.534 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:47:03.534 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:03.535 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:03.535 INFO:teuthology.orchestra.run.smithi101.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T02:47:03.535 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:03.535 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:03.536 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:03.536 INFO:teuthology.orchestra.run.smithi101.stdout:Install 1 Package 2023-01-25T02:47:03.536 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:03.536 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 82 k 2023-01-25T02:47:03.536 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 374 k 2023-01-25T02:47:03.537 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:03.826 INFO:teuthology.orchestra.run.smithi101.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 282 kB/s | 82 kB 00:00 2023-01-25T02:47:03.827 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:03.828 INFO:teuthology.orchestra.run.smithi101.stdout:Total 281 kB/s | 82 kB 00:00 2023-01-25T02:47:03.828 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:03.835 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:03.835 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:03.837 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:03.840 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:03.899 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:04.033 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:47:04.143 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:04.144 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:47:04.144 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:04.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install cephadm 2023-01-25T02:47:04.986 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:16 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:05.449 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:05.596 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:05.829 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:06.213 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:06.214 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:47:06.214 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:06.214 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:47:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T02:47:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:06.215 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:47:06.215 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:06.215 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-01-25T02:47:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:06.216 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 82 k 2023-01-25T02:47:06.216 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 374 k 2023-01-25T02:47:06.216 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:47:06.307 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:06.307 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:06.307 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:06.308 INFO:teuthology.orchestra.run.smithi101.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:06.308 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:06.308 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:06.389 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 471 kB/s | 82 kB 00:00 2023-01-25T02:47:06.390 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:06.391 INFO:teuthology.orchestra.run.smithi019.stdout:Total 468 kB/s | 82 kB 00:00 2023-01-25T02:47:06.391 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:47:06.399 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:47:06.399 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:47:06.401 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:47:06.404 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:47:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:47:06.550 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T02:47:07.282 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:20 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:08.159 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:08.336 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:08.424 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:08.425 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:08.425 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repo Size 2023-01-25T02:47:08.425 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:08.426 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:08.426 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T02:47:08.426 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:08.426 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:08.426 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:08.427 INFO:teuthology.orchestra.run.smithi101.stdout:Install 1 Package 2023-01-25T02:47:08.427 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:08.427 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 166 k 2023-01-25T02:47:08.427 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 503 k 2023-01-25T02:47:08.427 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:08.583 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:09.101 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:09.102 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:09.123 INFO:teuthology.orchestra.run.smithi101.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 238 kB/s | 166 kB 00:00 2023-01-25T02:47:09.123 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:09.124 INFO:teuthology.orchestra.run.smithi101.stdout:Total 237 kB/s | 166 kB 00:00 2023-01-25T02:47:09.124 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:09.137 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:09.137 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:09.192 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:09.195 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:09.867 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:10.024 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T02:47:10.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T02:47:10.464 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T02:47:10.896 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:22 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:10.943 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T02:47:10.943 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:10.944 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:10.944 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:47:10.944 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:10.944 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:11.159 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-mgr 2023-01-25T02:47:11.886 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:24 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:12.121 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:12.122 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:12.122 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-01-25T02:47:12.122 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:12.122 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:47:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T02:47:12.123 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:12.123 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:47:12.123 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:12.123 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-01-25T02:47:12.124 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:12.124 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 166 k 2023-01-25T02:47:12.124 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 503 k 2023-01-25T02:47:12.124 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:47:12.330 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 800 kB/s | 166 kB 00:00 2023-01-25T02:47:12.331 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:12.331 INFO:teuthology.orchestra.run.smithi019.stdout:Total 795 kB/s | 166 kB 00:00 2023-01-25T02:47:12.331 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:47:12.343 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:47:12.343 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:47:12.396 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:47:12.399 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:47:12.531 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:47:12.696 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T02:47:12.921 INFO:teuthology.orchestra.run.smithi101.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:47:13.021 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:13.022 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:47:13.022 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:13.116 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T02:47:13.151 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T02:47:13.657 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T02:47:13.658 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:13.658 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:13.658 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:47:13.658 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:13.658 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:13.838 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:26 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:13.852 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr 2023-01-25T02:47:14.603 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:26 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:14.978 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:14.979 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:14.979 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:47:14.979 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:14.979 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:14.979 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T02:47:14.980 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-01-25T02:47:14.980 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T02:47:14.980 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T02:47:14.980 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T02:47:14.980 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T02:47:14.981 INFO:teuthology.orchestra.run.smithi101.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T02:47:14.981 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:14.981 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:14.981 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:14.982 INFO:teuthology.orchestra.run.smithi101.stdout:Install 6 Packages 2023-01-25T02:47:14.982 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:14.982 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 2.0 M 2023-01-25T02:47:14.983 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 8.2 M 2023-01-25T02:47:14.983 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:15.396 INFO:teuthology.orchestra.run.smithi101.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 56 kB/s | 23 kB 00:00 2023-01-25T02:47:15.671 INFO:teuthology.orchestra.run.smithi101.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 21 kB/s | 15 kB 00:00 2023-01-25T02:47:15.730 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:47:15.830 INFO:teuthology.orchestra.run.smithi101.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 98 kB/s | 43 kB 00:00 2023-01-25T02:47:15.839 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:15.840 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:47:15.841 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:15.855 INFO:teuthology.orchestra.run.smithi101.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 183 kB/s | 34 kB 00:00 2023-01-25T02:47:15.897 INFO:teuthology.orchestra.run.smithi101.stdout:(5/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 1.9 MB/s | 1.7 MB 00:00 2023-01-25T02:47:16.056 INFO:teuthology.orchestra.run.smithi101.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 872 kB/s | 196 kB 00:00 2023-01-25T02:47:16.058 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:16.059 INFO:teuthology.orchestra.run.smithi101.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2023-01-25T02:47:16.072 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:16.081 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:16.081 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:16.169 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:16.173 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:16.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T02:47:16.438 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:16.662 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T02:47:16.857 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T02:47:17.034 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T02:47:17.127 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:29 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:17.176 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T02:47:17.543 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T02:47:17.736 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T02:47:18.018 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T02:47:18.019 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T02:47:18.019 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T02:47:18.019 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T02:47:18.019 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T02:47:18.020 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T02:47:18.356 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:18.357 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:18.357 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:47:18.358 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:18.358 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:47:18.358 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T02:47:18.358 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-01-25T02:47:18.358 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T02:47:18.359 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T02:47:18.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T02:47:18.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T02:47:18.359 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T02:47:18.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:18.360 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:47:18.360 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:18.360 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2023-01-25T02:47:18.360 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:18.361 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.0 M 2023-01-25T02:47:18.361 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 8.2 M 2023-01-25T02:47:18.362 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:47:18.511 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T02:47:18.511 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:18.512 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:18.512 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:18.512 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:18.512 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:18.513 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T02:47:18.513 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T02:47:18.513 INFO:teuthology.orchestra.run.smithi101.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T02:47:18.513 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:18.513 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:18.517 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 146 kB/s | 23 kB 00:00 2023-01-25T02:47:18.534 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 83 kB/s | 15 kB 00:00 2023-01-25T02:47:18.585 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 635 kB/s | 43 kB 00:00 2023-01-25T02:47:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 335 kB/s | 34 kB 00:00 2023-01-25T02:47:18.655 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T02:47:18.702 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 5.0 MB/s | 1.7 MB 00:00 2023-01-25T02:47:18.806 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 888 kB/s | 196 kB 00:00 2023-01-25T02:47:18.808 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:18.809 INFO:teuthology.orchestra.run.smithi019.stdout:Total 4.5 MB/s | 2.0 MB 00:00 2023-01-25T02:47:18.819 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:47:18.829 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:47:18.829 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:47:18.917 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:47:18.920 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:47:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:47:19.370 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:32 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T02:47:19.731 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T02:47:19.906 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T02:47:20.018 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T02:47:20.408 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T02:47:20.515 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:20.517 INFO:teuthology.orchestra.run.smithi101.stdout:===================================================================================================== 2023-01-25T02:47:20.517 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:47:20.517 INFO:teuthology.orchestra.run.smithi101.stdout:===================================================================================================== 2023-01-25T02:47:20.517 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:20.517 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T02:47:20.518 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-01-25T02:47:20.518 INFO:teuthology.orchestra.run.smithi101.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T02:47:20.518 INFO:teuthology.orchestra.run.smithi101.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T02:47:20.518 INFO:teuthology.orchestra.run.smithi101.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T02:47:20.518 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T02:47:20.519 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T02:47:20.519 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T02:47:20.519 INFO:teuthology.orchestra.run.smithi101.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T02:47:20.519 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:20.519 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:20.520 INFO:teuthology.orchestra.run.smithi101.stdout:===================================================================================================== 2023-01-25T02:47:20.520 INFO:teuthology.orchestra.run.smithi101.stdout:Install 8 Packages 2023-01-25T02:47:20.520 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:20.521 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 35 M 2023-01-25T02:47:20.521 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 210 M 2023-01-25T02:47:20.521 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:20.584 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T02:47:20.736 INFO:teuthology.orchestra.run.smithi101.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-01-25T02:47:20.936 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T02:47:20.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T02:47:20.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T02:47:20.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T02:47:20.938 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T02:47:20.938 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T02:47:20.939 INFO:teuthology.orchestra.run.smithi101.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 19 MB/s | 3.7 MB 00:00 2023-01-25T02:47:20.970 INFO:teuthology.orchestra.run.smithi101.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2023-01-25T02:47:21.288 INFO:teuthology.orchestra.run.smithi101.stdout:(4/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 6.3 MB/s | 4.9 MB 00:00 2023-01-25T02:47:21.396 INFO:teuthology.orchestra.run.smithi101.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 5.8 MB/s | 645 kB 00:00 2023-01-25T02:47:21.458 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T02:47:21.458 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:21.459 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:21.459 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:21.459 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:21.459 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:21.459 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T02:47:21.460 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T02:47:21.460 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T02:47:21.460 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:21.460 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:21.497 INFO:teuthology.orchestra.run.smithi101.stdout:(6/8): ceph-mgr-diskprediction-local-18.0.0-195 7.6 MB/s | 7.4 MB 00:00 2023-01-25T02:47:21.522 INFO:teuthology.orchestra.run.smithi101.stdout:(7/8): libquadmath-8.5.0-18.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2023-01-25T02:47:21.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T02:47:21.697 INFO:teuthology.orchestra.run.smithi101.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 19 MB/s | 14 MB 00:00 2023-01-25T02:47:21.699 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:21.699 INFO:teuthology.orchestra.run.smithi101.stdout:Total 30 MB/s | 35 MB 00:01 2023-01-25T02:47:21.929 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:21.947 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:21.947 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:22.321 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:22.338 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:22.342 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:22.904 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:23.047 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T02:47:23.181 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T02:47:23.322 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T02:47:23.561 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:23.562 INFO:teuthology.orchestra.run.smithi019.stdout:===================================================================================================== 2023-01-25T02:47:23.562 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:47:23.562 INFO:teuthology.orchestra.run.smithi019.stdout:===================================================================================================== 2023-01-25T02:47:23.563 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:47:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T02:47:23.563 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-01-25T02:47:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T02:47:23.563 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T02:47:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T02:47:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T02:47:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T02:47:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T02:47:23.564 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T02:47:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:23.565 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:47:23.565 INFO:teuthology.orchestra.run.smithi019.stdout:===================================================================================================== 2023-01-25T02:47:23.565 INFO:teuthology.orchestra.run.smithi019.stdout:Install 8 Packages 2023-01-25T02:47:23.565 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:23.566 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 35 M 2023-01-25T02:47:23.566 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 210 M 2023-01-25T02:47:23.566 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:47:23.796 INFO:teuthology.orchestra.run.smithi019.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-01-25T02:47:23.895 INFO:teuthology.orchestra.run.smithi019.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 15 MB/s | 4.9 MB 00:00 2023-01-25T02:47:23.921 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T02:47:23.929 INFO:teuthology.orchestra.run.smithi019.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.4 MB/s | 225 kB 00:00 2023-01-25T02:47:24.038 INFO:teuthology.orchestra.run.smithi019.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 15 MB/s | 3.7 MB 00:00 2023-01-25T02:47:24.088 INFO:teuthology.orchestra.run.smithi019.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 13 MB/s | 645 kB 00:00 2023-01-25T02:47:24.105 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T02:47:24.122 INFO:teuthology.orchestra.run.smithi019.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 5.0 MB/s | 172 kB 00:00 2023-01-25T02:47:24.422 INFO:teuthology.orchestra.run.smithi019.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 28 MB/s | 14 MB 00:00 2023-01-25T02:47:24.606 INFO:teuthology.orchestra.run.smithi019.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-195 7.1 MB/s | 7.4 MB 00:01 2023-01-25T02:47:24.609 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:24.609 INFO:teuthology.orchestra.run.smithi019.stdout:Total 34 MB/s | 35 MB 00:01 2023-01-25T02:47:24.727 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T02:47:24.849 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:47:24.868 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:47:24.869 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:47:24.913 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T02:47:25.292 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:47:25.296 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:47:25.602 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T02:47:25.878 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:47:25.941 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T02:47:26.019 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T02:47:26.140 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T02:47:26.146 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T02:47:26.269 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T02:47:26.905 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T02:47:27.337 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T02:47:27.924 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T02:47:28.056 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T02:47:28.235 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T02:47:28.998 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T02:47:29.184 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T02:47:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T02:47:29.371 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T02:47:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T02:47:29.701 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T02:47:29.701 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T02:47:29.702 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T02:47:29.702 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T02:47:29.702 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T02:47:29.702 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T02:47:29.702 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T02:47:29.703 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T02:47:30.218 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T02:47:30.218 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:30.218 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:30.218 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:30.219 INFO:teuthology.orchestra.run.smithi101.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T02:47:30.219 INFO:teuthology.orchestra.run.smithi101.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T02:47:30.219 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T02:47:30.219 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T02:47:30.220 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T02:47:30.220 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T02:47:30.220 INFO:teuthology.orchestra.run.smithi101.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T02:47:30.220 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:30.220 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:30.326 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-mgr-rook 2023-01-25T02:47:31.053 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:44 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T02:47:32.197 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:32.198 INFO:teuthology.orchestra.run.smithi101.stdout:======================================================================================== 2023-01-25T02:47:32.198 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:47:32.199 INFO:teuthology.orchestra.run.smithi101.stdout:======================================================================================== 2023-01-25T02:47:32.199 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:32.199 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T02:47:32.199 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-01-25T02:47:32.199 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T02:47:32.200 INFO:teuthology.orchestra.run.smithi101.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T02:47:32.200 INFO:teuthology.orchestra.run.smithi101.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T02:47:32.200 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T02:47:32.200 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T02:47:32.200 INFO:teuthology.orchestra.run.smithi101.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T02:47:32.201 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T02:47:32.201 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T02:47:32.201 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T02:47:32.201 INFO:teuthology.orchestra.run.smithi101.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T02:47:32.201 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T02:47:32.202 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T02:47:32.202 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:32.202 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:32.202 INFO:teuthology.orchestra.run.smithi101.stdout:======================================================================================== 2023-01-25T02:47:32.202 INFO:teuthology.orchestra.run.smithi101.stdout:Install 13 Packages 2023-01-25T02:47:32.203 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:32.203 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 1.9 M 2023-01-25T02:47:32.203 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 25 M 2023-01-25T02:47:32.204 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:32.287 INFO:teuthology.orchestra.run.smithi101.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 313 kB/s | 27 kB 00:00 2023-01-25T02:47:32.312 INFO:teuthology.orchestra.run.smithi101.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 181 kB/s | 20 kB 00:00 2023-01-25T02:47:32.337 INFO:teuthology.orchestra.run.smithi101.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-25T02:47:32.363 INFO:teuthology.orchestra.run.smithi101.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-25T02:47:32.388 INFO:teuthology.orchestra.run.smithi101.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-01-25T02:47:32.413 INFO:teuthology.orchestra.run.smithi101.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 850 kB/s | 43 kB 00:00 2023-01-25T02:47:32.438 INFO:teuthology.orchestra.run.smithi101.stdout:(7/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 200 kB/s | 47 kB 00:00 2023-01-25T02:47:32.547 INFO:teuthology.orchestra.run.smithi101.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 122 kB/s | 16 kB 00:00 2023-01-25T02:47:32.572 INFO:teuthology.orchestra.run.smithi101.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 208 kB/s | 33 kB 00:00 2023-01-25T02:47:32.598 INFO:teuthology.orchestra.run.smithi101.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 511 kB/s | 81 kB 00:00 2023-01-25T02:47:32.665 INFO:teuthology.orchestra.run.smithi101.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 668 kB/s | 61 kB 00:00 2023-01-25T02:47:32.681 INFO:teuthology.orchestra.run.smithi101.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 733 kB/s | 61 kB 00:00 2023-01-25T02:47:32.745 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T02:47:33.182 INFO:teuthology.orchestra.run.smithi101.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.8 MB/s | 1.2 MB 00:00 2023-01-25T02:47:33.183 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:33.184 INFO:teuthology.orchestra.run.smithi101.stdout:Total 2.0 MB/s | 1.9 MB 00:00 2023-01-25T02:47:33.237 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:33.252 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:33.252 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:33.349 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:33.353 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:33.855 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:33.868 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T02:47:34.148 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T02:47:34.231 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T02:47:34.232 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T02:47:34.232 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T02:47:34.232 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T02:47:34.232 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T02:47:34.233 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T02:47:34.233 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T02:47:34.233 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T02:47:34.346 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T02:47:34.589 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T02:47:34.778 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T02:47:34.778 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:34.779 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:34.779 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:34.779 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T02:47:34.779 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T02:47:34.779 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T02:47:34.780 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T02:47:34.780 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T02:47:34.780 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T02:47:34.780 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T02:47:34.780 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:34.781 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:34.860 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T02:47:34.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-rook 2023-01-25T02:47:35.034 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T02:47:35.248 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T02:47:35.581 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T02:47:35.663 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:47 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:35.832 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T02:47:36.885 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:36.887 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2023-01-25T02:47:36.887 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:47:36.888 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2023-01-25T02:47:36.888 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:47:36.888 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T02:47:36.888 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-01-25T02:47:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T02:47:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T02:47:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T02:47:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T02:47:36.889 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T02:47:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T02:47:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T02:47:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T02:47:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T02:47:36.890 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T02:47:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T02:47:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T02:47:36.891 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:36.891 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:47:36.891 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================== 2023-01-25T02:47:36.892 INFO:teuthology.orchestra.run.smithi019.stdout:Install 13 Packages 2023-01-25T02:47:36.892 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:36.892 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 1.9 M 2023-01-25T02:47:36.893 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 25 M 2023-01-25T02:47:36.893 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:47:36.914 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T02:47:36.956 INFO:teuthology.orchestra.run.smithi019.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 307 kB/s | 20 kB 00:00 2023-01-25T02:47:36.981 INFO:teuthology.orchestra.run.smithi019.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 294 kB/s | 27 kB 00:00 2023-01-25T02:47:37.007 INFO:teuthology.orchestra.run.smithi019.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-25T02:47:37.032 INFO:teuthology.orchestra.run.smithi019.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-25T02:47:37.058 INFO:teuthology.orchestra.run.smithi019.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-01-25T02:47:37.074 INFO:teuthology.orchestra.run.smithi019.stdout:(6/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 258 kB/s | 47 kB 00:00 2023-01-25T02:47:37.091 INFO:teuthology.orchestra.run.smithi019.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 724 kB/s | 43 kB 00:00 2023-01-25T02:47:37.167 INFO:teuthology.orchestra.run.smithi019.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 177 kB/s | 16 kB 00:00 2023-01-25T02:47:37.177 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T02:47:37.192 INFO:teuthology.orchestra.run.smithi019.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 245 kB/s | 33 kB 00:00 2023-01-25T02:47:37.218 INFO:teuthology.orchestra.run.smithi019.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 646 kB/s | 81 kB 00:00 2023-01-25T02:47:37.268 INFO:teuthology.orchestra.run.smithi019.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 817 kB/s | 61 kB 00:00 2023-01-25T02:47:37.318 INFO:teuthology.orchestra.run.smithi019.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 611 kB/s | 61 kB 00:00 2023-01-25T02:47:37.360 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T02:47:37.537 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T02:47:37.877 INFO:teuthology.orchestra.run.smithi019.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.6 MB/s | 1.2 MB 00:00 2023-01-25T02:47:37.878 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:37.879 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.9 MB/s | 1.9 MB 00:00 2023-01-25T02:47:37.938 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:47:37.954 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:47:37.954 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:47:38.061 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:47:38.064 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:47:38.512 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T02:47:38.588 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:47:38.830 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T02:47:38.831 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T02:47:38.831 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T02:47:38.831 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T02:47:38.831 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T02:47:38.831 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T02:47:38.832 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T02:47:38.832 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T02:47:38.832 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T02:47:38.832 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T02:47:38.832 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T02:47:38.833 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T02:47:38.833 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T02:47:38.873 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T02:47:39.071 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T02:47:39.296 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T02:47:39.382 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T02:47:39.383 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:39.383 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:39.383 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:39.384 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T02:47:39.384 INFO:teuthology.orchestra.run.smithi101.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T02:47:39.384 INFO:teuthology.orchestra.run.smithi101.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T02:47:39.384 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T02:47:39.384 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T02:47:39.385 INFO:teuthology.orchestra.run.smithi101.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T02:47:39.385 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T02:47:39.385 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T02:47:39.385 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T02:47:39.385 INFO:teuthology.orchestra.run.smithi101.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T02:47:39.386 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T02:47:39.386 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T02:47:39.386 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:39.386 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:39.510 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T02:47:39.876 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T02:47:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T02:47:40.227 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:53 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:40.310 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T02:47:40.615 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T02:47:40.884 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T02:47:41.366 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:41.367 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:41.367 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:47:41.367 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:41.367 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:41.368 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T02:47:41.368 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-01-25T02:47:41.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T02:47:41.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T02:47:41.368 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T02:47:41.369 INFO:teuthology.orchestra.run.smithi101.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T02:47:41.369 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:41.369 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:41.369 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:41.369 INFO:teuthology.orchestra.run.smithi101.stdout:Install 5 Packages 2023-01-25T02:47:41.370 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:41.370 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 6.0 M 2023-01-25T02:47:41.370 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 26 M 2023-01-25T02:47:41.371 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:41.464 INFO:teuthology.orchestra.run.smithi101.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.6 MB/s | 538 kB 00:00 2023-01-25T02:47:41.622 INFO:teuthology.orchestra.run.smithi101.stdout:(2/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 475 kB/s | 120 kB 00:00 2023-01-25T02:47:41.689 INFO:teuthology.orchestra.run.smithi101.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-01-25T02:47:41.823 INFO:teuthology.orchestra.run.smithi101.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 301 kB/s | 60 kB 00:00 2023-01-25T02:47:41.906 INFO:teuthology.orchestra.run.smithi101.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.1 MB/s | 476 kB 00:00 2023-01-25T02:47:41.908 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:41.908 INFO:teuthology.orchestra.run.smithi101.stdout:Total 11 MB/s | 6.0 MB 00:00 2023-01-25T02:47:41.959 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:41.970 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:41.970 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:42.027 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T02:47:42.108 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:42.111 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:42.315 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T02:47:42.390 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:42.566 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T02:47:42.709 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T02:47:42.744 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T02:47:42.900 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T02:47:43.219 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T02:47:43.219 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T02:47:43.219 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T02:47:43.219 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T02:47:43.220 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T02:47:43.220 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T02:47:43.220 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T02:47:43.220 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T02:47:43.220 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T02:47:43.221 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T02:47:43.221 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T02:47:43.221 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T02:47:43.221 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T02:47:43.655 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T02:47:43.803 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T02:47:43.803 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:43.804 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:43.804 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T02:47:43.804 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T02:47:43.804 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T02:47:43.804 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T02:47:43.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T02:47:43.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T02:47:43.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T02:47:43.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T02:47:43.805 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T02:47:43.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T02:47:43.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T02:47:43.806 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T02:47:43.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:43.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T02:47:44.031 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T02:47:44.297 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T02:47:44.486 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T02:47:44.685 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:56 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:44.788 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T02:47:44.788 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T02:47:44.788 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T02:47:44.788 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T02:47:44.789 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T02:47:45.289 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T02:47:45.289 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:45.289 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:45.290 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:45.290 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T02:47:45.290 INFO:teuthology.orchestra.run.smithi101.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T02:47:45.290 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T02:47:45.290 INFO:teuthology.orchestra.run.smithi101.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T02:47:45.291 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:45.291 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:45.391 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-fuse 2023-01-25T02:47:45.907 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:45.908 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:45.909 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:47:45.909 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:45.909 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:47:45.909 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T02:47:45.909 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-01-25T02:47:45.910 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T02:47:45.910 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T02:47:45.910 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T02:47:45.910 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T02:47:45.910 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:45.911 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:47:45.911 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:45.911 INFO:teuthology.orchestra.run.smithi019.stdout:Install 5 Packages 2023-01-25T02:47:45.911 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:45.912 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 6.0 M 2023-01-25T02:47:45.912 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 26 M 2023-01-25T02:47:45.912 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:47:46.002 INFO:teuthology.orchestra.run.smithi019.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.8 MB/s | 538 kB 00:00 2023-01-25T02:47:46.119 INFO:teuthology.orchestra.run.smithi019.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-01-25T02:47:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:59 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:46.144 INFO:teuthology.orchestra.run.smithi019.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 516 kB/s | 120 kB 00:00 2023-01-25T02:47:46.253 INFO:teuthology.orchestra.run.smithi019.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 451 kB/s | 60 kB 00:00 2023-01-25T02:47:46.378 INFO:teuthology.orchestra.run.smithi019.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.2 MB/s | 476 kB 00:00 2023-01-25T02:47:46.379 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:46.380 INFO:teuthology.orchestra.run.smithi019.stdout:Total 13 MB/s | 6.0 MB 00:00 2023-01-25T02:47:46.433 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:47:46.445 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:47:46.445 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:47:46.587 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:47:46.591 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:47:46.877 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:47:47.223 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T02:47:47.265 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:47.266 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:47.266 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:47:47.267 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:47.267 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:47.267 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T02:47:47.267 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-01-25T02:47:47.267 INFO:teuthology.orchestra.run.smithi101.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T02:47:47.268 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:47.268 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:47.268 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:47.268 INFO:teuthology.orchestra.run.smithi101.stdout:Install 2 Packages 2023-01-25T02:47:47.268 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:47.269 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 928 k 2023-01-25T02:47:47.269 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 2.9 M 2023-01-25T02:47:47.270 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:47.405 INFO:teuthology.orchestra.run.smithi101.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 604 kB/s | 83 kB 00:00 2023-01-25T02:47:47.964 INFO:teuthology.orchestra.run.smithi101.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 1.2 MB/s | 845 kB 00:00 2023-01-25T02:47:47.965 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:47.965 INFO:teuthology.orchestra.run.smithi101.stdout:Total 1.3 MB/s | 928 kB 00:00 2023-01-25T02:47:47.976 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:47.997 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:47.998 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:48.091 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:48.094 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:48.217 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T02:47:48.295 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:48.550 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T02:47:48.619 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T02:47:48.767 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T02:47:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T02:47:49.075 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T02:47:49.196 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T02:47:49.197 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T02:47:49.397 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T02:47:49.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T02:47:49.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T02:47:49.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T02:47:49.398 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T02:47:49.686 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T02:47:49.686 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:49.687 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:49.687 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T02:47:49.687 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:49.687 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:49.892 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-volume 2023-01-25T02:47:49.927 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T02:47:49.927 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:49.928 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:49.928 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:49.928 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T02:47:49.928 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T02:47:49.928 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T02:47:49.929 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T02:47:49.929 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:49.929 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:50.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-fuse 2023-01-25T02:47:50.634 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:03 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:50.792 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:02 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:51.767 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:51.768 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:51.768 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-01-25T02:47:51.768 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:51.769 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:51.769 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T02:47:51.769 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:51.769 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:51.769 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:51.770 INFO:teuthology.orchestra.run.smithi101.stdout:Install 1 Package 2023-01-25T02:47:51.770 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:51.770 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 263 k 2023-01-25T02:47:51.770 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 1.3 M 2023-01-25T02:47:51.770 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:52.013 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:52.013 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:52.014 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:47:52.014 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:52.014 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:47:52.014 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T02:47:52.014 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2023-01-25T02:47:52.015 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T02:47:52.015 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:52.015 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:47:52.015 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:52.015 INFO:teuthology.orchestra.run.smithi019.stdout:Install 2 Packages 2023-01-25T02:47:52.016 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:52.016 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 928 k 2023-01-25T02:47:52.017 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 2.9 M 2023-01-25T02:47:52.017 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:47:52.042 INFO:teuthology.orchestra.run.smithi101.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 963 kB/s | 263 kB 00:00 2023-01-25T02:47:52.042 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:52.043 INFO:teuthology.orchestra.run.smithi101.stdout:Total 959 kB/s | 263 kB 00:00 2023-01-25T02:47:52.043 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:52.054 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:52.054 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:52.104 INFO:teuthology.orchestra.run.smithi019.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 935 kB/s | 83 kB 00:00 2023-01-25T02:47:52.148 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:52.152 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:52.336 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:52.478 INFO:teuthology.orchestra.run.smithi019.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 1.8 MB/s | 845 kB 00:00 2023-01-25T02:47:52.479 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:52.479 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.0 MB/s | 928 kB 00:00 2023-01-25T02:47:52.490 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:47:52.512 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:47:52.513 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:47:52.610 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:52.613 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:47:52.617 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:47:52.832 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:47:53.050 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:53.094 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T02:47:53.290 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T02:47:53.541 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:53.541 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:53.541 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:53.541 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:53.541 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:53.542 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:53.703 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T02:47:53.703 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T02:47:53.756 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install librados-devel 2023-01-25T02:47:54.271 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T02:47:54.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:54.272 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:54.272 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T02:47:54.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:54.273 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:54.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-volume 2023-01-25T02:47:54.495 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:07 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:55.236 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:07 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:55.644 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:55.645 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:55.645 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repo Size 2023-01-25T02:47:55.645 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:55.645 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:47:55.646 INFO:teuthology.orchestra.run.smithi101.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T02:47:55.646 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:55.646 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:47:55.646 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:47:55.646 INFO:teuthology.orchestra.run.smithi101.stdout:Install 1 Package 2023-01-25T02:47:55.647 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:55.647 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 139 k 2023-01-25T02:47:55.647 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 512 k 2023-01-25T02:47:55.647 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:47:55.855 INFO:teuthology.orchestra.run.smithi101.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 666 kB/s | 139 kB 00:00 2023-01-25T02:47:55.855 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:55.856 INFO:teuthology.orchestra.run.smithi101.stdout:Total 662 kB/s | 139 kB 00:00 2023-01-25T02:47:55.856 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:47:55.861 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:47:55.861 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:47:55.913 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:47:55.916 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:47:56.052 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:47:56.248 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:47:56.435 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:47:56.436 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:56.437 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2023-01-25T02:47:56.437 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:56.437 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:47:56.437 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T02:47:56.437 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:56.437 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:47:56.438 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:47:56.438 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-01-25T02:47:56.438 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:56.438 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 263 k 2023-01-25T02:47:56.439 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 1.3 M 2023-01-25T02:47:56.439 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:47:56.545 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:47:56.699 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 1.0 MB/s | 263 kB 00:00 2023-01-25T02:47:56.700 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:47:56.701 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.0 MB/s | 263 kB 00:00 2023-01-25T02:47:56.701 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:47:56.711 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:47:56.712 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:47:56.796 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:47:56.799 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:47:56.992 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:47:57.014 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:47:57.014 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:57.014 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:47:57.015 INFO:teuthology.orchestra.run.smithi101.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:47:57.015 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:47:57.015 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:57.225 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install libcephfs2 2023-01-25T02:47:57.257 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:57.721 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:57.960 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:10 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:47:58.196 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T02:47:58.197 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:47:58.197 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T02:47:58.197 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:47:58.197 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:47:58.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados-devel 2023-01-25T02:47:58.994 INFO:teuthology.orchestra.run.smithi101.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:47:59.095 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:47:59.096 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:47:59.096 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:47:59.147 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:11 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:47:59.195 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install libcephfs-devel 2023-01-25T02:47:59.916 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:12 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:00.373 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:00.374 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:00.374 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-01-25T02:48:00.374 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:00.374 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:48:00.375 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T02:48:00.375 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:00.375 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:48:00.375 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:00.376 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-01-25T02:48:00.376 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:00.376 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 139 k 2023-01-25T02:48:00.376 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 512 k 2023-01-25T02:48:00.376 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:48:00.974 INFO:teuthology.orchestra.run.smithi019.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 232 kB/s | 139 kB 00:00 2023-01-25T02:48:00.974 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:00.975 INFO:teuthology.orchestra.run.smithi019.stdout:Total 232 kB/s | 139 kB 00:00 2023-01-25T02:48:00.975 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:48:00.982 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:48:00.982 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:48:01.047 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:48:01.051 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:48:01.053 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:01.054 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:01.054 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repo Size 2023-01-25T02:48:01.054 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:01.055 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:48:01.055 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T02:48:01.055 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:01.055 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:48:01.055 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:01.056 INFO:teuthology.orchestra.run.smithi101.stdout:Install 1 Package 2023-01-25T02:48:01.056 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:01.057 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 30 k 2023-01-25T02:48:01.057 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 137 k 2023-01-25T02:48:01.057 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:48:01.210 INFO:teuthology.orchestra.run.smithi101.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 197 kB/s | 30 kB 00:00 2023-01-25T02:48:01.211 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:01.212 INFO:teuthology.orchestra.run.smithi101.stdout:Total 195 kB/s | 30 kB 00:00 2023-01-25T02:48:01.212 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:48:01.215 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:48:01.215 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:48:01.239 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:48:01.242 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:48:01.308 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:48:01.497 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:48:01.798 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:02.258 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:02.258 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:02.258 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:48:02.259 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:02.259 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:02.259 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:02.390 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install librados2 2023-01-25T02:48:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:02.724 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:48:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:02.724 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:02.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs2 2023-01-25T02:48:03.110 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:16 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:03.689 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:15 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:04.151 INFO:teuthology.orchestra.run.smithi101.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:04.250 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:04.251 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:48:04.251 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:04.351 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install librbd1 2023-01-25T02:48:04.799 INFO:teuthology.orchestra.run.smithi019.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:04.911 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:04.912 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:48:04.912 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:04.998 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs-devel 2023-01-25T02:48:05.074 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:18 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:05.760 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:17 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:06.110 INFO:teuthology.orchestra.run.smithi101.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:06.210 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:06.211 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:48:06.211 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:06.311 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install python3-rados 2023-01-25T02:48:06.982 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:06.983 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-01-25T02:48:06.983 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:06.983 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:48:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T02:48:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:06.984 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:48:06.984 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:06.984 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-01-25T02:48:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 30 k 2023-01-25T02:48:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 137 k 2023-01-25T02:48:06.985 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:48:07.034 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:20 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 43 kB/s | 30 kB 00:00 2023-01-25T02:48:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:Total 43 kB/s | 30 kB 00:00 2023-01-25T02:48:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:48:07.698 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:48:07.698 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:48:07.733 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:48:07.737 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:48:07.805 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:48:07.987 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:08.077 INFO:teuthology.orchestra.run.smithi101.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:08.176 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:08.177 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:48:08.177 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:08.277 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install python3-rgw 2023-01-25T02:48:08.287 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:08.780 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:08.780 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:08.780 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:48:08.781 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:08.781 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:08.781 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:08.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados2 2023-01-25T02:48:09.001 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:22 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:09.642 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:21 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:10.047 INFO:teuthology.orchestra.run.smithi101.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:10.148 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:10.149 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:48:10.149 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:10.239 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install python3-cephfs 2023-01-25T02:48:10.765 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:10.878 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:10.880 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:48:10.880 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:10.959 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:23 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:10.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librbd1 2023-01-25T02:48:11.731 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:23 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:12.001 INFO:teuthology.orchestra.run.smithi101.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:12.101 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:12.102 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:48:12.103 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:12.221 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install python3-rbd 2023-01-25T02:48:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:12.954 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:12.961 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:12.964 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:48:12.964 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:13.047 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rados 2023-01-25T02:48:13.807 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:13.995 INFO:teuthology.orchestra.run.smithi101.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:14.097 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:14.098 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-01-25T02:48:14.098 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:14.196 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install rbd-fuse 2023-01-25T02:48:14.915 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:27 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:14.921 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:15.032 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:15.033 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:48:15.034 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:15.124 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rgw 2023-01-25T02:48:15.885 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:27 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:16.056 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:16.057 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:16.057 INFO:teuthology.orchestra.run.smithi101.stdout: Package Architecture Version Repository Size 2023-01-25T02:48:16.057 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:16.058 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:48:16.058 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T02:48:16.058 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:16.058 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:48:16.058 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:16.059 INFO:teuthology.orchestra.run.smithi101.stdout:Install 1 Package 2023-01-25T02:48:16.059 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:16.059 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 94 k 2023-01-25T02:48:16.059 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 280 k 2023-01-25T02:48:16.059 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:48:16.795 INFO:teuthology.orchestra.run.smithi101.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 128 kB/s | 94 kB 00:00 2023-01-25T02:48:16.796 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:16.796 INFO:teuthology.orchestra.run.smithi101.stdout:Total 128 kB/s | 94 kB 00:00 2023-01-25T02:48:16.797 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:48:16.807 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:48:16.807 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:48:16.862 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:48:16.865 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:48:16.989 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:48:17.010 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:17.122 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:17.123 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:48:17.123 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:17.196 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:17.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-cephfs 2023-01-25T02:48:17.488 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:17.964 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:29 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:18.004 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:18.004 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:18.005 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:48:18.005 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:18.005 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:18.005 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:18.207 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install rbd-mirror 2023-01-25T02:48:18.936 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:31 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:19.096 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:19.210 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:19.212 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:48:19.212 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:19.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rbd 2023-01-25T02:48:20.072 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:32 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:20.074 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:20.075 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:20.075 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repo Size 2023-01-25T02:48:20.075 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:20.076 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:48:20.076 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T02:48:20.076 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:20.076 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:48:20.077 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:20.077 INFO:teuthology.orchestra.run.smithi101.stdout:Install 1 Package 2023-01-25T02:48:20.077 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:20.078 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 3.4 M 2023-01-25T02:48:20.078 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 16 M 2023-01-25T02:48:20.078 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:48:21.179 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T02:48:21.291 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:21.292 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2023-01-25T02:48:21.293 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:21.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-fuse 2023-01-25T02:48:22.141 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:34 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:22.778 INFO:teuthology.orchestra.run.smithi101.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 1.3 MB/s | 3.4 MB 00:02 2023-01-25T02:48:22.778 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:22.779 INFO:teuthology.orchestra.run.smithi101.stdout:Total 1.3 MB/s | 3.4 MB 00:02 2023-01-25T02:48:22.779 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:48:22.793 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:48:22.793 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:48:22.862 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:48:22.865 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:48:23.305 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:48:23.370 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:23.371 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2023-01-25T02:48:23.372 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:23.372 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:48:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T02:48:23.372 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:23.373 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:48:23.373 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:23.373 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-01-25T02:48:23.373 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:23.373 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 94 k 2023-01-25T02:48:23.374 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 280 k 2023-01-25T02:48:23.374 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:48:23.523 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:23.569 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 480 kB/s | 94 kB 00:00 2023-01-25T02:48:23.570 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:23.570 INFO:teuthology.orchestra.run.smithi019.stdout:Total 476 kB/s | 94 kB 00:00 2023-01-25T02:48:23.571 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:48:23.582 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:48:23.582 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:48:23.637 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:48:23.640 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:48:23.786 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:48:23.969 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:24.001 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:24.313 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:24.481 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:24.481 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:24.482 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:48:24.482 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:24.482 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:24.482 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:24.666 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install rbd-nbd 2023-01-25T02:48:24.808 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:24.808 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:24.808 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:48:24.809 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:24.809 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:24.809 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:25.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-mirror 2023-01-25T02:48:25.406 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:38 ago on Wed 25 Jan 2023 02:45:47 AM UTC. 2023-01-25T02:48:25.772 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:37 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:26.549 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-01-25T02:48:26.550 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:26.550 INFO:teuthology.orchestra.run.smithi101.stdout: Package Architecture Version Repository Size 2023-01-25T02:48:26.550 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:26.550 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-01-25T02:48:26.551 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T02:48:26.551 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:26.551 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-01-25T02:48:26.551 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-01-25T02:48:26.551 INFO:teuthology.orchestra.run.smithi101.stdout:Install 1 Package 2023-01-25T02:48:26.552 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:26.552 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 179 k 2023-01-25T02:48:26.552 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 568 k 2023-01-25T02:48:26.552 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-01-25T02:48:26.788 INFO:teuthology.orchestra.run.smithi101.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 756 kB/s | 179 kB 00:00 2023-01-25T02:48:26.789 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:26.789 INFO:teuthology.orchestra.run.smithi101.stdout:Total 752 kB/s | 179 kB 00:00 2023-01-25T02:48:26.790 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-01-25T02:48:26.800 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-01-25T02:48:26.801 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-01-25T02:48:26.853 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-01-25T02:48:26.856 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-01-25T02:48:26.979 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-01-25T02:48:27.007 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:27.008 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:27.008 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2023-01-25T02:48:27.008 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:27.008 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:48:27.009 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T02:48:27.009 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:27.009 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:48:27.009 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:27.009 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-01-25T02:48:27.010 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:27.010 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 3.4 M 2023-01-25T02:48:27.010 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 16 M 2023-01-25T02:48:27.011 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:48:27.161 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:27.483 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:27.848 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 4.1 MB/s | 3.4 MB 00:00 2023-01-25T02:48:27.849 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:27.849 INFO:teuthology.orchestra.run.smithi019.stdout:Total 4.1 MB/s | 3.4 MB 00:00 2023-01-25T02:48:27.849 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:48:27.863 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:48:27.863 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:48:27.935 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:48:27.939 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:48:27.942 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:27.942 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:27.943 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-01-25T02:48:27.943 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:27.943 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:27.943 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-01-25T02:48:28.146 DEBUG:teuthology.parallel:result is None 2023-01-25T02:48:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:48:28.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:29.106 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:29.662 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:29.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:29.663 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:48:29.663 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:29.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:29.664 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:29.851 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-nbd 2023-01-25T02:48:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:02:42 ago on Wed 25 Jan 2023 02:45:48 AM UTC. 2023-01-25T02:48:31.865 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2023-01-25T02:48:31.866 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:31.866 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2023-01-25T02:48:31.867 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:31.867 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2023-01-25T02:48:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T02:48:31.867 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:31.867 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2023-01-25T02:48:31.868 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2023-01-25T02:48:31.868 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2023-01-25T02:48:31.868 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:31.868 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 179 k 2023-01-25T02:48:31.868 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 568 k 2023-01-25T02:48:31.869 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2023-01-25T02:48:32.069 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 889 kB/s | 179 kB 00:00 2023-01-25T02:48:32.070 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2023-01-25T02:48:32.070 INFO:teuthology.orchestra.run.smithi019.stdout:Total 884 kB/s | 179 kB 00:00 2023-01-25T02:48:32.070 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2023-01-25T02:48:32.082 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2023-01-25T02:48:32.083 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2023-01-25T02:48:32.139 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2023-01-25T02:48:32.142 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2023-01-25T02:48:32.267 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2023-01-25T02:48:32.442 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:32.761 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:33.287 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T02:48:33.287 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2023-01-25T02:48:33.288 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T02:48:33.288 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:33.288 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2023-01-25T02:48:33.470 DEBUG:teuthology.parallel:result is None 2023-01-25T02:48:33.471 DEBUG:teuthology.orchestra.run.smithi019:> sudo lsb_release -is 2023-01-25T02:48:33.509 INFO:teuthology.orchestra.run.smithi019.stdout:CentOSStream 2023-01-25T02:48:33.510 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T02:48:33.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T02:48:33.782 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T02:48:33.806 INFO:teuthology.orchestra.run.smithi019.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T02:48:33.806 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T02:48:33.807 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T02:48:33.807 DEBUG:teuthology.orchestra.run.smithi101:> sudo lsb_release -is 2023-01-25T02:48:33.847 INFO:teuthology.orchestra.run.smithi101.stdout:CentOSStream 2023-01-25T02:48:33.848 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T02:48:33.849 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T02:48:34.135 DEBUG:teuthology.orchestra.run.smithi101:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T02:48:34.158 INFO:teuthology.orchestra.run.smithi101.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T02:48:34.159 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T02:48:34.159 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T02:48:34.159 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T02:48:34.160 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:34.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T02:48:34.192 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:34.192 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T02:48:34.226 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T02:48:34.226 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:34.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T02:48:34.263 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T02:48:34.333 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:34.334 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T02:48:34.361 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T02:48:34.428 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T02:48:34.428 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:34.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T02:48:34.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T02:48:34.531 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:34.531 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T02:48:34.558 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T02:48:34.625 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T02:48:34.693 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T02:48:34.693 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /var/log/ceph 2023-01-25T02:48:34.696 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 777 /var/log/ceph 2023-01-25T02:48:34.726 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T02:48:34.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T02:48:34.768 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T02:48:34.797 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T02:48:34.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T02:48:34.840 DEBUG:teuthology.orchestra.run.smithi101:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T02:48:34.880 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T02:48:34.881 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd sloppy crc': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-25T02:48:34.881 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136602', 'branch': 'wip-yuri-testing-2023-01-23-0926', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7136602', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-01-23-0926'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd sloppy crc': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(TOO_FEW_PGS\\)', 'but it is still running', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(AUTH_BAD_CAPS\\)'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd objectstore': 'filestore', 'osd sloppy crc': True}}, 'filestore': True, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-yuri-testing-2023-01-23-0926', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri-testing-2023-01-23-0926', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'targets': {'smithi019.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCWKmrrkTJTYAqtSJfveg6gKmkIyR25CC3odKcn/Pnr9ILns6nNCPhgUo/FRjOMuOh17ZjFBO+HciyrIE5YheYsDvZc7IFjlKGcyKGUwdsxg2xmgWSqPEfcqLjaUjcJfir5GS9Y5PoKwkWAOBQULL1zBQePfn8N7TC771Utwg5bup8KSzaFefUiGemUhLPr1VVURzHXx3Ecjqowoz3bp8yRnqi1NpvP7d5kyYxupBVyp8Rbg8IIqzx0n5+wYdLPw/kSdM7vsCywpbkkrkMBWa4t2ImvQnIuhFlBX0yRqdNYkeynGSNhdwoUII2a02LUIC+QDS4Q1MHHqrOUg1hsILKbJzdhaxJ9jstAOzZjvioNDWaiXVWEpIAJRcFrbObPIUty2LCMNpzGlrkp1MT4L7R6/XvBrTCPW8tc9pLsGPn65szhBVZg3SZ3Mx6hO4Evvq9iSyRDVvjyeFeKYp9n5NW1oxDGvNZsN3qXTeK7fhdJTjvaBe8o7YNmu6D0e8Rm1M=', 'smithi101.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0sgtEPttGl4btdP1NXT+jebLPumtSqYs6wL2/vMovcojm5YIkaKGFQptgjglOFfEQwxzzVqdU3TlsJnaPkOo7+0c8diSvmmwAKFUp1mHgXk8hyRzIoDzA9yqwlulI4bL0Ekm0DZKqd5vgi9iI4l7xFTGfOOggQu8BOqHpvyBQ+c1vHm4bkXeI6Ke2ZQyud60g0ewuX/gIrLzNkwQNbQ+pEZYoSU/4k9RjmwfE3utfWyDmXkoR0Vhp/vw1GUakB6/cYyub1OOSXeus5wHuUT6g8xVFhb7nR3sVZqGXFXemhsYJ5u/AMhZqQD0c0RqOy+2HzSYCBWaGnfB/Lth5uRZxcuKGp8ZixCmKP89w/ecIc0kb9vbYydNiqRVgaK1dHFp2TyAXlsOg0kf12IQsMw2Hj/2++7RC2IaOUoHJF/KtnkzEM68BuUr1VHBYd9fdAa5yTQzrklBWCPKUT/UxxaM2FhOuVlCB9/tqP7HIHS5qr6pRUphdvn4KVxNUo4Op28='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'store_thrash': True, 'thrash_delay': 1, 'thrash_many': True}}, {'workunit': {'clients': {'client.0': ['mon/pool_ops.sh', 'mon/crush_ops.sh', 'mon/osd.sh', 'mon/caps.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:20:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T02:48:34.882 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T02:48:34.911 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T02:48:34.929 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T02:48:34.971 DEBUG:teuthology.orchestra.run.smithi101:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T02:48:35.000 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:35.000 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T02:48:35.060 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T02:48:35.060 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2023-01-25T02:48:35.119 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T02:48:35.120 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T02:48:35.120 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 375085 Links: 1 2023-01-25T02:48:35.120 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T02:48:35.121 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T02:48:35.121 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2023-01-25 02:48:28.833876074 +0000 2023-01-25T02:48:35.121 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2023-01-25 02:45:01.061074691 +0000 2023-01-25T02:48:35.121 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2023-01-25 02:45:01.061074691 +0000 2023-01-25T02:48:35.121 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2023-01-25T02:48:35.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T02:48:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2023-01-25T02:48:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2023-01-25T02:48:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000236983 s, 2.2 MB/s 2023-01-25T02:48:35.189 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T02:48:35.249 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2023-01-25T02:48:35.309 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T02:48:35.310 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T02:48:35.310 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 371138 Links: 1 2023-01-25T02:48:35.310 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T02:48:35.310 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T02:48:35.311 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2023-01-25 02:48:28.833876074 +0000 2023-01-25T02:48:35.311 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2023-01-25 02:45:01.579069842 +0000 2023-01-25T02:48:35.311 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2023-01-25 02:45:01.579069842 +0000 2023-01-25T02:48:35.311 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2023-01-25T02:48:35.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T02:48:35.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2023-01-25T02:48:35.379 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2023-01-25T02:48:35.379 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000200204 s, 2.6 MB/s 2023-01-25T02:48:35.380 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T02:48:35.441 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2023-01-25T02:48:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T02:48:35.501 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T02:48:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 375205 Links: 1 2023-01-25T02:48:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T02:48:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T02:48:35.503 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2023-01-25 02:48:28.833876074 +0000 2023-01-25T02:48:35.503 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2023-01-25 02:45:02.077065188 +0000 2023-01-25T02:48:35.503 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2023-01-25 02:45:02.077065188 +0000 2023-01-25T02:48:35.503 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2023-01-25T02:48:35.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T02:48:35.571 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2023-01-25T02:48:35.571 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2023-01-25T02:48:35.572 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.0002017 s, 2.5 MB/s 2023-01-25T02:48:35.573 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T02:48:35.634 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2023-01-25T02:48:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T02:48:35.694 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T02:48:35.694 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 375980 Links: 1 2023-01-25T02:48:35.694 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T02:48:35.695 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T02:48:35.695 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2023-01-25 02:48:28.833876074 +0000 2023-01-25T02:48:35.695 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2023-01-25 02:45:02.603060272 +0000 2023-01-25T02:48:35.695 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2023-01-25 02:45:02.603060272 +0000 2023-01-25T02:48:35.695 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: - 2023-01-25T02:48:35.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T02:48:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2023-01-25T02:48:35.762 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2023-01-25T02:48:35.763 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000187117 s, 2.7 MB/s 2023-01-25T02:48:35.764 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T02:48:35.824 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-25T02:48:35.825 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:35.825 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T02:48:35.844 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T02:48:35.844 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_1 2023-01-25T02:48:35.901 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T02:48:35.902 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T02:48:35.902 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 366537 Links: 1 2023-01-25T02:48:35.902 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T02:48:35.902 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T02:48:35.903 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-01-25 02:48:23.702908527 +0000 2023-01-25T02:48:35.903 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-01-25 02:44:50.566108922 +0000 2023-01-25T02:48:35.903 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-01-25 02:44:50.566108922 +0000 2023-01-25T02:48:35.903 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2023-01-25T02:48:35.903 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T02:48:35.968 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-01-25T02:48:35.968 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-01-25T02:48:35.968 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000216378 s, 2.4 MB/s 2023-01-25T02:48:35.969 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T02:48:36.028 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_2 2023-01-25T02:48:36.085 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T02:48:36.086 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T02:48:36.086 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 366573 Links: 1 2023-01-25T02:48:36.086 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T02:48:36.086 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T02:48:36.087 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-01-25 02:48:23.702908527 +0000 2023-01-25T02:48:36.087 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-01-25 02:44:51.075097544 +0000 2023-01-25T02:48:36.087 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-01-25 02:44:51.075097544 +0000 2023-01-25T02:48:36.087 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2023-01-25T02:48:36.087 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T02:48:36.151 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-01-25T02:48:36.152 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-01-25T02:48:36.152 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000214352 s, 2.4 MB/s 2023-01-25T02:48:36.153 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T02:48:36.211 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_3 2023-01-25T02:48:36.270 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T02:48:36.270 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T02:48:36.270 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 367213 Links: 1 2023-01-25T02:48:36.270 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T02:48:36.271 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T02:48:36.271 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-01-25 02:48:23.702908527 +0000 2023-01-25T02:48:36.271 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-01-25 02:44:51.577086322 +0000 2023-01-25T02:48:36.271 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-01-25 02:44:51.577086322 +0000 2023-01-25T02:48:36.272 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2023-01-25T02:48:36.272 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T02:48:36.335 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-01-25T02:48:36.336 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-01-25T02:48:36.336 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000216753 s, 2.4 MB/s 2023-01-25T02:48:36.337 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T02:48:36.396 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_4 2023-01-25T02:48:36.454 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T02:48:36.455 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T02:48:36.455 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 6h/6d Inode: 369362 Links: 1 2023-01-25T02:48:36.455 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T02:48:36.455 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T02:48:36.456 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-01-25 02:48:23.702908527 +0000 2023-01-25T02:48:36.456 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-01-25 02:44:52.240071502 +0000 2023-01-25T02:48:36.456 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-01-25 02:44:52.240071502 +0000 2023-01-25T02:48:36.456 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: - 2023-01-25T02:48:36.457 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T02:48:36.520 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-01-25T02:48:36.521 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-01-25T02:48:36.521 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000209704 s, 2.4 MB/s 2023-01-25T02:48:36.522 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T02:48:36.580 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2023-01-25T02:48:36.581 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi101.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2023-01-25T02:48:36.581 INFO:tasks.ceph:Generating config... 2023-01-25T02:48:36.582 INFO:tasks.ceph:[client] debug monc = 20 2023-01-25T02:48:36.583 INFO:tasks.ceph:[client] debug ms = 1 2023-01-25T02:48:36.583 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-25T02:48:36.583 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-25T02:48:36.584 INFO:tasks.ceph:[global] ms bind msgr1 = False 2023-01-25T02:48:36.584 INFO:tasks.ceph:[global] ms bind msgr2 = True 2023-01-25T02:48:36.584 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-01-25T02:48:36.584 INFO:tasks.ceph:[global] ms type = async 2023-01-25T02:48:36.585 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T02:48:36.585 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T02:48:36.585 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T02:48:36.585 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T02:48:36.586 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T02:48:36.586 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2023-01-25T02:48:36.586 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-25T02:48:36.587 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-25T02:48:36.587 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-25T02:48:36.587 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2023-01-25T02:48:36.588 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2023-01-25T02:48:36.588 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T02:48:36.588 INFO:tasks.ceph:[mon] paxos service trim min = 5 2023-01-25T02:48:36.588 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T02:48:36.589 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T02:48:36.589 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T02:48:36.589 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T02:48:36.589 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T02:48:36.590 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T02:48:36.590 INFO:tasks.ceph:[osd] osd objectstore = filestore 2023-01-25T02:48:36.590 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T02:48:36.590 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T02:48:36.591 INFO:tasks.ceph:[osd] osd sloppy crc = True 2023-01-25T02:48:36.591 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T02:48:36.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:36.656 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T02:48:36.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T02:48:36.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T02:48:36.736 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.19'), ('mon.b', '[v2:172.21.15.19:3301,v1:172.21.15.19:6790]'), ('mon.c', '[v2:172.21.15.19:3302,v1:172.21.15.19:6791]'), ('mon.d', '[v2:172.21.15.19:3303,v1:172.21.15.19:6792]'), ('mon.e', '[v2:172.21.15.19:3304,v1:172.21.15.19:6793]'), ('mon.f', '172.21.15.101'), ('mon.g', '[v2:172.21.15.101:3301,v1:172.21.15.101:6790]'), ('mon.h', '[v2:172.21.15.101:3302,v1:172.21.15.101:6791]'), ('mon.i', '[v2:172.21.15.101:3303,v1:172.21.15.101:6792]')] 2023-01-25T02:48:36.736 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],[v2:172.21.15.19:3302,v1:172.21.15.19:6791],[v2:172.21.15.19:3303,v1:172.21.15.19:6792],[v2:172.21.15.19:3304,v1:172.21.15.19:6793],172.21.15.101,[v2:172.21.15.101:3301,v1:172.21.15.101:6790],[v2:172.21.15.101:3302,v1:172.21.15.101:6791],[v2:172.21.15.101:3303,v1:172.21.15.101:6792]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd objectstore': 'filestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd sloppy crc': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos service trim min': 5}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2023-01-25T02:48:36.737 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:36.737 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T02:48:36.795 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.19 --addv b '[v2:172.21.15.19:3301,v1:172.21.15.19:6790]' --addv c '[v2:172.21.15.19:3302,v1:172.21.15.19:6791]' --addv d '[v2:172.21.15.19:3303,v1:172.21.15.19:6792]' --addv e '[v2:172.21.15.19:3304,v1:172.21.15.19:6793]' --add f 172.21.15.101 --addv g '[v2:172.21.15.101:3301,v1:172.21.15.101:6790]' --addv h '[v2:172.21.15.101:3302,v1:172.21.15.101:6791]' --addv i '[v2:172.21.15.101:3303,v1:172.21.15.101:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:36.905 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:36.905 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: generated fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:36.906 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = pacific 2023-01-25T02:48:36.906 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 0 2023-01-25T02:48:36.906 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:36.906 INFO:teuthology.orchestra.run.smithi019.stdout:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:36.906 INFO:teuthology.orchestra.run.smithi019.stdout:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:36.906 INFO:teuthology.orchestra.run.smithi019.stdout:min_mon_release 16 (pacific) 2023-01-25T02:48:36.907 INFO:teuthology.orchestra.run.smithi019.stdout:election_strategy: 3 2023-01-25T02:48:36.907 INFO:teuthology.orchestra.run.smithi019.stdout:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:36.907 INFO:teuthology.orchestra.run.smithi019.stdout:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:36.907 INFO:teuthology.orchestra.run.smithi019.stdout:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:36.907 INFO:teuthology.orchestra.run.smithi019.stdout:3: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:36.907 INFO:teuthology.orchestra.run.smithi019.stdout:4: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:36.908 INFO:teuthology.orchestra.run.smithi019.stdout:5: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:36.908 INFO:teuthology.orchestra.run.smithi019.stdout:6: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:36.908 INFO:teuthology.orchestra.run.smithi019.stdout:7: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:36.909 INFO:teuthology.orchestra.run.smithi019.stdout:8: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:36.909 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2023-01-25T02:48:36.910 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T02:48:36.928 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 079d300e-8dda-42d1-8c3c-224268975a31... 2023-01-25T02:48:36.929 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T02:48:36.970 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T02:48:37.015 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2023-01-25T02:48:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2023-01-25T02:48:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T02:48:37.015 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2023-01-25T02:48:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2023-01-25T02:48:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2023-01-25T02:48:37.016 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2023-01-25T02:48:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2023-01-25T02:48:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.017 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2023-01-25T02:48:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2023-01-25T02:48:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T02:48:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2023-01-25T02:48:37.018 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T02:48:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T02:48:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2023-01-25T02:48:37.019 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2023-01-25T02:48:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2023-01-25T02:48:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2023-01-25T02:48:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2023-01-25T02:48:37.020 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2023-01-25T02:48:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2023-01-25T02:48:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2023-01-25T02:48:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T02:48:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T02:48:37.021 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2023-01-25T02:48:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T02:48:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2023-01-25T02:48:37.022 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2023-01-25T02:48:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2023-01-25T02:48:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2023-01-25T02:48:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2023-01-25T02:48:37.023 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],[v2:172.21.15.19:3302,v1:172.21.15.19:6791],[v2:172.21.15.19:3303,v1:172.21.15.19:6792],[v2:172.21.15.19:3304,v1:172.21.15.19:6793],172.21.15.101,[v2:172.21.15.101:3301,v1:172.21.15.101:6790],[v2:172.21.15.101:3302,v1:172.21.15.101:6791],[v2:172.21.15.101:3303,v1:172.21.15.101:6792]" 2023-01-25T02:48:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: mon client directed command retry = 5 2023-01-25T02:48:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: mon election default strategy = 3 2023-01-25T02:48:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: ms bind msgr1 = False 2023-01-25T02:48:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: ms bind msgr2 = True 2023-01-25T02:48:37.024 INFO:teuthology.orchestra.run.smithi019.stdout: ms inject socket failures = 5000 2023-01-25T02:48:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: ms type = async 2023-01-25T02:48:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.025 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2023-01-25T02:48:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2023-01-25T02:48:37.025 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2023-01-25T02:48:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2023-01-25T02:48:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2023-01-25T02:48:37.026 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2023-01-25T02:48:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2023-01-25T02:48:37.027 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2023-01-25T02:48:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2023-01-25T02:48:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2023-01-25T02:48:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2023-01-25T02:48:37.028 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2023-01-25T02:48:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2023-01-25T02:48:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.029 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2023-01-25T02:48:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2023-01-25T02:48:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2023-01-25T02:48:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2023-01-25T02:48:37.030 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2023-01-25T02:48:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore zero block detection = True 2023-01-25T02:48:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2023-01-25T02:48:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2023-01-25T02:48:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify cached snaps = True 2023-01-25T02:48:37.031 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify missing on start = True 2023-01-25T02:48:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock override recovery settings = True 2023-01-25T02:48:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = filestore 2023-01-25T02:48:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: osd op queue = debug_random 2023-01-25T02:48:37.032 INFO:teuthology.orchestra.run.smithi019.stdout: osd op queue cut off = debug_random 2023-01-25T02:48:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd sloppy crc = True 2023-01-25T02:48:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.033 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2023-01-25T02:48:37.033 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2023-01-25T02:48:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2023-01-25T02:48:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2023-01-25T02:48:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2023-01-25T02:48:37.034 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2023-01-25T02:48:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2023-01-25T02:48:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2023-01-25T02:48:37.035 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.035 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2023-01-25T02:48:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2023-01-25T02:48:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2023-01-25T02:48:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2023-01-25T02:48:37.036 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2023-01-25T02:48:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2023-01-25T02:48:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2023-01-25T02:48:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2023-01-25T02:48:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2023-01-25T02:48:37.037 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2023-01-25T02:48:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2023-01-25T02:48:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T02:48:37.038 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T02:48:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T02:48:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T02:48:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T02:48:37.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T02:48:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2023-01-25T02:48:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2023-01-25T02:48:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.040 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2023-01-25T02:48:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: mon min osdmap epochs = 25 2023-01-25T02:48:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune interval = 2 2023-01-25T02:48:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune min = 15 2023-01-25T02:48:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune txsize = 2 2023-01-25T02:48:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: mon scrub inject crc mismatch = 0.01 2023-01-25T02:48:37.041 INFO:teuthology.orchestra.run.smithi019.stdout: mon scrub inject missing keys = 0.05 2023-01-25T02:48:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: mon scrub interval = 300 2023-01-25T02:48:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: paxos service trim min = 5 2023-01-25T02:48:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:48:37.042 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2023-01-25T02:48:37.042 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2023-01-25T02:48:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2023-01-25T02:48:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2023-01-25T02:48:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T02:48:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T02:48:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: debug monc = 20 2023-01-25T02:48:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2023-01-25T02:48:37.044 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2023-01-25T02:48:37.044 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2023-01-25T02:48:37.044 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2023-01-25T02:48:37.045 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.d] 2023-01-25T02:48:37.045 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.e] 2023-01-25T02:48:37.045 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.f] 2023-01-25T02:48:37.045 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.g] 2023-01-25T02:48:37.046 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.h] 2023-01-25T02:48:37.046 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.i] 2023-01-25T02:48:37.047 INFO:teuthology.orchestra.run.smithi101.stdout:[global] 2023-01-25T02:48:37.048 INFO:teuthology.orchestra.run.smithi101.stdout: chdir = "" 2023-01-25T02:48:37.048 INFO:teuthology.orchestra.run.smithi101.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T02:48:37.048 INFO:teuthology.orchestra.run.smithi101.stdout: auth supported = cephx 2023-01-25T02:48:37.048 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.048 INFO:teuthology.orchestra.run.smithi101.stdout: filestore xattr use omap = true 2023-01-25T02:48:37.049 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.049 INFO:teuthology.orchestra.run.smithi101.stdout: mon clock drift allowed = 1.000 2023-01-25T02:48:37.049 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.049 INFO:teuthology.orchestra.run.smithi101.stdout: osd crush chooseleaf type = 0 2023-01-25T02:48:37.049 INFO:teuthology.orchestra.run.smithi101.stdout: auth debug = true 2023-01-25T02:48:37.050 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.050 INFO:teuthology.orchestra.run.smithi101.stdout: ms die on old message = true 2023-01-25T02:48:37.050 INFO:teuthology.orchestra.run.smithi101.stdout: ms die on bug = true 2023-01-25T02:48:37.050 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.050 INFO:teuthology.orchestra.run.smithi101.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T02:48:37.050 INFO:teuthology.orchestra.run.smithi101.stdout: mon pg warn max object skew = 0 2023-01-25T02:48:37.051 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.051 INFO:teuthology.orchestra.run.smithi101.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T02:48:37.051 INFO:teuthology.orchestra.run.smithi101.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T02:48:37.051 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.051 INFO:teuthology.orchestra.run.smithi101.stdout: osd pool default size = 2 2023-01-25T02:48:37.051 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.052 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd allow primary affinity = true 2023-01-25T02:48:37.052 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd allow pg remap = true 2023-01-25T02:48:37.052 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on legacy crush tunables = false 2023-01-25T02:48:37.052 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on crush straw calc version zero = false 2023-01-25T02:48:37.052 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on no sortbitwise = false 2023-01-25T02:48:37.053 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on osd down out interval zero = false 2023-01-25T02:48:37.053 INFO:teuthology.orchestra.run.smithi101.stdout: mon warn on too few osds = false 2023-01-25T02:48:37.053 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T02:48:37.053 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T02:48:37.053 INFO:teuthology.orchestra.run.smithi101.stdout: mon_allow_pool_size_one = true 2023-01-25T02:48:37.054 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.054 INFO:teuthology.orchestra.run.smithi101.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T02:48:37.054 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.054 INFO:teuthology.orchestra.run.smithi101.stdout: osd default data pool replay window = 5 2023-01-25T02:48:37.054 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.055 INFO:teuthology.orchestra.run.smithi101.stdout: mon allow pool delete = true 2023-01-25T02:48:37.055 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.055 INFO:teuthology.orchestra.run.smithi101.stdout: mon cluster log file level = debug 2023-01-25T02:48:37.055 INFO:teuthology.orchestra.run.smithi101.stdout: debug asserts on shutdown = true 2023-01-25T02:48:37.055 INFO:teuthology.orchestra.run.smithi101.stdout: mon health detail to clog = false 2023-01-25T02:48:37.055 INFO:teuthology.orchestra.run.smithi101.stdout: mon host = "172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],[v2:172.21.15.19:3302,v1:172.21.15.19:6791],[v2:172.21.15.19:3303,v1:172.21.15.19:6792],[v2:172.21.15.19:3304,v1:172.21.15.19:6793],172.21.15.101,[v2:172.21.15.101:3301,v1:172.21.15.101:6790],[v2:172.21.15.101:3302,v1:172.21.15.101:6791],[v2:172.21.15.101:3303,v1:172.21.15.101:6792]" 2023-01-25T02:48:37.056 INFO:teuthology.orchestra.run.smithi101.stdout: mon client directed command retry = 5 2023-01-25T02:48:37.056 INFO:teuthology.orchestra.run.smithi101.stdout: mon election default strategy = 3 2023-01-25T02:48:37.056 INFO:teuthology.orchestra.run.smithi101.stdout: ms bind msgr1 = False 2023-01-25T02:48:37.056 INFO:teuthology.orchestra.run.smithi101.stdout: ms bind msgr2 = True 2023-01-25T02:48:37.057 INFO:teuthology.orchestra.run.smithi101.stdout: ms inject socket failures = 5000 2023-01-25T02:48:37.057 INFO:teuthology.orchestra.run.smithi101.stdout: ms type = async 2023-01-25T02:48:37.057 INFO:teuthology.orchestra.run.smithi101.stdout: fsid = 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:37.057 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.057 INFO:teuthology.orchestra.run.smithi101.stdout:[osd] 2023-01-25T02:48:37.057 INFO:teuthology.orchestra.run.smithi101.stdout: osd journal size = 100 2023-01-25T02:48:37.058 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.058 INFO:teuthology.orchestra.run.smithi101.stdout: osd scrub load threshold = 5.0 2023-01-25T02:48:37.058 INFO:teuthology.orchestra.run.smithi101.stdout: osd scrub max interval = 600 2023-01-25T02:48:37.058 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.058 INFO:teuthology.orchestra.run.smithi101.stdout: osd recover clone overlap = true 2023-01-25T02:48:37.058 INFO:teuthology.orchestra.run.smithi101.stdout: osd recovery max chunk = 1048576 2023-01-25T02:48:37.059 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.059 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug shutdown = true 2023-01-25T02:48:37.059 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug op order = true 2023-01-25T02:48:37.059 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug verify stray on activate = true 2023-01-25T02:48:37.059 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.060 INFO:teuthology.orchestra.run.smithi101.stdout: osd open classes on start = true 2023-01-25T02:48:37.060 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug pg log writeout = true 2023-01-25T02:48:37.060 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.060 INFO:teuthology.orchestra.run.smithi101.stdout: osd deep scrub update digest min age = 30 2023-01-25T02:48:37.060 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.060 INFO:teuthology.orchestra.run.smithi101.stdout: osd map max advance = 10 2023-01-25T02:48:37.061 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.061 INFO:teuthology.orchestra.run.smithi101.stdout: journal zero on create = true 2023-01-25T02:48:37.061 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.061 INFO:teuthology.orchestra.run.smithi101.stdout: filestore ondisk finisher threads = 3 2023-01-25T02:48:37.061 INFO:teuthology.orchestra.run.smithi101.stdout: filestore apply finisher threads = 3 2023-01-25T02:48:37.061 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.062 INFO:teuthology.orchestra.run.smithi101.stdout: bdev debug aio = true 2023-01-25T02:48:37.062 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug misdirected ops = true 2023-01-25T02:48:37.062 INFO:teuthology.orchestra.run.smithi101.stdout: bluestore zero block detection = True 2023-01-25T02:48:37.062 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 1 2023-01-25T02:48:37.062 INFO:teuthology.orchestra.run.smithi101.stdout: debug osd = 20 2023-01-25T02:48:37.063 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug verify cached snaps = True 2023-01-25T02:48:37.063 INFO:teuthology.orchestra.run.smithi101.stdout: osd debug verify missing on start = True 2023-01-25T02:48:37.063 INFO:teuthology.orchestra.run.smithi101.stdout: osd mclock override recovery settings = True 2023-01-25T02:48:37.063 INFO:teuthology.orchestra.run.smithi101.stdout: osd objectstore = filestore 2023-01-25T02:48:37.063 INFO:teuthology.orchestra.run.smithi101.stdout: osd op queue = debug_random 2023-01-25T02:48:37.064 INFO:teuthology.orchestra.run.smithi101.stdout: osd op queue cut off = debug_random 2023-01-25T02:48:37.064 INFO:teuthology.orchestra.run.smithi101.stdout: osd sloppy crc = True 2023-01-25T02:48:37.064 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.064 INFO:teuthology.orchestra.run.smithi101.stdout:[mgr] 2023-01-25T02:48:37.064 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 1 2023-01-25T02:48:37.064 INFO:teuthology.orchestra.run.smithi101.stdout: debug mgr = 20 2023-01-25T02:48:37.065 INFO:teuthology.orchestra.run.smithi101.stdout: debug mon = 20 2023-01-25T02:48:37.065 INFO:teuthology.orchestra.run.smithi101.stdout: debug auth = 20 2023-01-25T02:48:37.065 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min pgs per osd = 4 2023-01-25T02:48:37.065 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min bytes per osd = 10 2023-01-25T02:48:37.065 INFO:teuthology.orchestra.run.smithi101.stdout: mgr/telemetry/nag = false 2023-01-25T02:48:37.066 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.066 INFO:teuthology.orchestra.run.smithi101.stdout:[mon] 2023-01-25T02:48:37.066 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 1 2023-01-25T02:48:37.066 INFO:teuthology.orchestra.run.smithi101.stdout: debug mon = 20 2023-01-25T02:48:37.066 INFO:teuthology.orchestra.run.smithi101.stdout: debug paxos = 20 2023-01-25T02:48:37.066 INFO:teuthology.orchestra.run.smithi101.stdout: debug auth = 20 2023-01-25T02:48:37.067 INFO:teuthology.orchestra.run.smithi101.stdout: mon data avail warn = 5 2023-01-25T02:48:37.067 INFO:teuthology.orchestra.run.smithi101.stdout: mon mgr mkfs grace = 240 2023-01-25T02:48:37.067 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min pgs per osd = 4 2023-01-25T02:48:37.067 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd reporter subtree level = osd 2023-01-25T02:48:37.067 INFO:teuthology.orchestra.run.smithi101.stdout: mon osd prime pg temp = true 2023-01-25T02:48:37.068 INFO:teuthology.orchestra.run.smithi101.stdout: mon reweight min bytes per osd = 10 2023-01-25T02:48:37.068 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.068 INFO:teuthology.orchestra.run.smithi101.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T02:48:37.068 INFO:teuthology.orchestra.run.smithi101.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T02:48:37.068 INFO:teuthology.orchestra.run.smithi101.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T02:48:37.068 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.069 INFO:teuthology.orchestra.run.smithi101.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T02:48:37.069 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T02:48:37.069 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T02:48:37.069 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.069 INFO:teuthology.orchestra.run.smithi101.stdout: # 1m isn't quite enough 2023-01-25T02:48:37.069 INFO:teuthology.orchestra.run.smithi101.stdout: mon_down_mkfs_grace = 2m 2023-01-25T02:48:37.070 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.070 INFO:teuthology.orchestra.run.smithi101.stdout: mon_warn_on_filestore_osds = false 2023-01-25T02:48:37.070 INFO:teuthology.orchestra.run.smithi101.stdout: mon min osdmap epochs = 25 2023-01-25T02:48:37.070 INFO:teuthology.orchestra.run.smithi101.stdout: mon osdmap full prune interval = 2 2023-01-25T02:48:37.070 INFO:teuthology.orchestra.run.smithi101.stdout: mon osdmap full prune min = 15 2023-01-25T02:48:37.071 INFO:teuthology.orchestra.run.smithi101.stdout: mon osdmap full prune txsize = 2 2023-01-25T02:48:37.071 INFO:teuthology.orchestra.run.smithi101.stdout: mon scrub inject crc mismatch = 0.01 2023-01-25T02:48:37.071 INFO:teuthology.orchestra.run.smithi101.stdout: mon scrub inject missing keys = 0.05 2023-01-25T02:48:37.071 INFO:teuthology.orchestra.run.smithi101.stdout: mon scrub interval = 300 2023-01-25T02:48:37.071 INFO:teuthology.orchestra.run.smithi101.stdout: paxos service trim min = 5 2023-01-25T02:48:37.072 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-01-25T02:48:37.072 INFO:teuthology.orchestra.run.smithi101.stdout:[client] 2023-01-25T02:48:37.072 INFO:teuthology.orchestra.run.smithi101.stdout: rgw cache enabled = true 2023-01-25T02:48:37.072 INFO:teuthology.orchestra.run.smithi101.stdout: rgw enable ops log = true 2023-01-25T02:48:37.072 INFO:teuthology.orchestra.run.smithi101.stdout: rgw enable usage log = true 2023-01-25T02:48:37.072 INFO:teuthology.orchestra.run.smithi101.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T02:48:37.073 INFO:teuthology.orchestra.run.smithi101.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T02:48:37.073 INFO:teuthology.orchestra.run.smithi101.stdout: debug monc = 20 2023-01-25T02:48:37.073 INFO:teuthology.orchestra.run.smithi101.stdout: debug ms = 1 2023-01-25T02:48:37.073 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.a] 2023-01-25T02:48:37.073 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.b] 2023-01-25T02:48:37.074 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.c] 2023-01-25T02:48:37.074 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.d] 2023-01-25T02:48:37.074 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.e] 2023-01-25T02:48:37.074 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.f] 2023-01-25T02:48:37.074 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.g] 2023-01-25T02:48:37.075 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.h] 2023-01-25T02:48:37.075 INFO:teuthology.orchestra.run.smithi101.stdout:[mon.i] 2023-01-25T02:48:37.075 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T02:48:37.075 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T02:48:37.145 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T02:48:37.146 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:37.146 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T02:48:37.164 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:37.165 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T02:48:37.222 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi019.front.sepia.ceph.com 2023-01-25T02:48:37.222 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:37.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T02:48:37.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T02:48:37.300 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:37.300 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:37.358 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi101.front.sepia.ceph.com 2023-01-25T02:48:37.358 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:37.359 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T02:48:37.359 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T02:48:37.396 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:37.396 DEBUG:teuthology.orchestra.run.smithi101:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:37.453 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T02:48:37.454 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T02:48:37.454 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T02:48:37.564 INFO:teuthology.orchestra.run.smithi101.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T02:48:37.567 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T02:48:37.567 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T02:48:37.567 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T02:48:37.611 INFO:teuthology.orchestra.run.smithi101.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T02:48:37.623 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T02:48:37.624 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi101.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2023-01-25T02:48:37.624 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T02:48:37.652 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-25T02:48:37.653 INFO:tasks.ceph:role: osd.0 2023-01-25T02:48:37.653 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com 2023-01-25T02:48:37.653 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T02:48:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T02:48:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T02:48:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T02:48:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T02:48:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T02:48:37.730 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2023-01-25T02:48:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T02:48:37.730 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T02:48:37.731 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T02:48:37.731 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T02:48:38.156 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2023-01-25T02:48:38.157 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2023-01-25T02:48:38.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T02:48:38.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T02:48:38.332 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T02:48:38.401 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-25T02:48:38.401 INFO:tasks.ceph:role: osd.1 2023-01-25T02:48:38.402 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com 2023-01-25T02:48:38.402 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T02:48:38.477 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T02:48:38.477 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T02:48:38.478 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T02:48:38.478 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T02:48:38.478 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T02:48:38.478 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2023-01-25T02:48:38.478 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T02:48:38.479 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T02:48:38.479 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T02:48:38.479 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T02:48:38.894 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2023-01-25T02:48:38.896 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2023-01-25T02:48:38.897 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T02:48:38.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T02:48:39.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T02:48:39.084 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-25T02:48:39.084 INFO:tasks.ceph:role: osd.2 2023-01-25T02:48:39.085 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com 2023-01-25T02:48:39.085 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T02:48:39.160 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T02:48:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T02:48:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T02:48:39.160 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T02:48:39.161 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T02:48:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2023-01-25T02:48:39.161 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T02:48:39.161 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T02:48:39.161 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T02:48:39.161 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T02:48:39.603 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2023-01-25T02:48:39.605 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2023-01-25T02:48:39.605 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T02:48:39.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T02:48:39.753 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:39.855 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:39.854+0000 7fa4708d25c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T02:48:39.858 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:39.856+0000 7fa4708d25c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T02:48:39.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:39.873+0000 7fa4708d25c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:39.874 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:39.873+0000 7fa4708d25c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T02:48:40.128 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.126+0000 7fa4708d25c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.128+0000 7fa4708d25c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:40.130 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.128+0000 7fa4708d25c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:40.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T02:48:40.186 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:40.275 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.274+0000 7f05aa6765c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T02:48:40.278 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.276+0000 7f05aa6765c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T02:48:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.302+0000 7f05aa6765c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:40.304 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.303+0000 7f05aa6765c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T02:48:40.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.546+0000 7f05aa6765c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:40.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.552+0000 7f05aa6765c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:40.554 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.552+0000 7f05aa6765c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:40.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T02:48:40.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:40.707 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.706+0000 7f14e30955c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T02:48:40.708 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.707+0000 7f14e30955c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T02:48:40.736 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.735+0000 7f14e30955c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:40.737 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.735+0000 7f14e30955c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T02:48:40.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.967+0000 7f14e30955c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:40.969 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.968+0000 7f14e30955c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:40.970 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:40.968+0000 7f14e30955c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:40.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T02:48:41.006 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-25T02:48:41.036 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2023-01-25T02:48:41.036 INFO:tasks.ceph:role: osd.3 2023-01-25T02:48:41.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi101.front.sepia.ceph.com 2023-01-25T02:48:41.037 DEBUG:teuthology.orchestra.run.smithi101:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T02:48:41.111 INFO:teuthology.orchestra.run.smithi101.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T02:48:41.112 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T02:48:41.112 INFO:teuthology.orchestra.run.smithi101.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T02:48:41.112 INFO:teuthology.orchestra.run.smithi101.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T02:48:41.112 INFO:teuthology.orchestra.run.smithi101.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T02:48:41.112 INFO:teuthology.orchestra.run.smithi101.stdout: = sunit=0 swidth=0 blks 2023-01-25T02:48:41.113 INFO:teuthology.orchestra.run.smithi101.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T02:48:41.113 INFO:teuthology.orchestra.run.smithi101.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T02:48:41.113 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T02:48:41.113 INFO:teuthology.orchestra.run.smithi101.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T02:48:41.544 INFO:teuthology.orchestra.run.smithi101.stdout:Discarding blocks...Done. 2023-01-25T02:48:41.545 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi101.front.sepia.ceph.com -o noatime 2023-01-25T02:48:41.545 DEBUG:teuthology.orchestra.run.smithi101:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2023-01-25T02:48:41.713 DEBUG:teuthology.orchestra.run.smithi101:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-25T02:48:41.750 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-25T02:48:41.819 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2023-01-25T02:48:41.819 INFO:tasks.ceph:role: osd.4 2023-01-25T02:48:41.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi101.front.sepia.ceph.com 2023-01-25T02:48:41.820 DEBUG:teuthology.orchestra.run.smithi101:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T02:48:41.894 INFO:teuthology.orchestra.run.smithi101.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T02:48:41.895 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T02:48:41.895 INFO:teuthology.orchestra.run.smithi101.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T02:48:41.895 INFO:teuthology.orchestra.run.smithi101.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T02:48:41.895 INFO:teuthology.orchestra.run.smithi101.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T02:48:41.895 INFO:teuthology.orchestra.run.smithi101.stdout: = sunit=0 swidth=0 blks 2023-01-25T02:48:41.896 INFO:teuthology.orchestra.run.smithi101.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T02:48:41.896 INFO:teuthology.orchestra.run.smithi101.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T02:48:41.896 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T02:48:41.896 INFO:teuthology.orchestra.run.smithi101.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T02:48:42.324 INFO:teuthology.orchestra.run.smithi101.stdout:Discarding blocks...Done. 2023-01-25T02:48:42.326 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi101.front.sepia.ceph.com -o noatime 2023-01-25T02:48:42.326 DEBUG:teuthology.orchestra.run.smithi101:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2023-01-25T02:48:42.440 DEBUG:teuthology.orchestra.run.smithi101:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-25T02:48:42.473 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-25T02:48:42.541 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2023-01-25T02:48:42.541 INFO:tasks.ceph:role: osd.5 2023-01-25T02:48:42.541 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi101.front.sepia.ceph.com 2023-01-25T02:48:42.542 DEBUG:teuthology.orchestra.run.smithi101:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T02:48:42.616 INFO:teuthology.orchestra.run.smithi101.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T02:48:42.616 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T02:48:42.616 INFO:teuthology.orchestra.run.smithi101.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T02:48:42.616 INFO:teuthology.orchestra.run.smithi101.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T02:48:42.617 INFO:teuthology.orchestra.run.smithi101.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T02:48:42.617 INFO:teuthology.orchestra.run.smithi101.stdout: = sunit=0 swidth=0 blks 2023-01-25T02:48:42.617 INFO:teuthology.orchestra.run.smithi101.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T02:48:42.617 INFO:teuthology.orchestra.run.smithi101.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T02:48:42.617 INFO:teuthology.orchestra.run.smithi101.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T02:48:42.617 INFO:teuthology.orchestra.run.smithi101.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T02:48:43.065 INFO:teuthology.orchestra.run.smithi101.stdout:Discarding blocks...Done. 2023-01-25T02:48:43.066 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi101.front.sepia.ceph.com -o noatime 2023-01-25T02:48:43.067 DEBUG:teuthology.orchestra.run.smithi101:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2023-01-25T02:48:43.221 DEBUG:teuthology.orchestra.run.smithi101:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-25T02:48:43.254 DEBUG:teuthology.orchestra.run.smithi101:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:43.352 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.351+0000 7f1b071435c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-25T02:48:43.355 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.355+0000 7f1b071435c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-25T02:48:43.381 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.380+0000 7f1b071435c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:43.382 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.381+0000 7f1b071435c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T02:48:43.631 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.630+0000 7f1b071435c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:43.634 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.634+0000 7f1b071435c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:43.635 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.634+0000 7f1b071435c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:43.660 DEBUG:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-25T02:48:43.687 DEBUG:teuthology.orchestra.run.smithi101:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:43.774 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.773+0000 7f5bf81e95c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-25T02:48:43.776 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.775+0000 7f5bf81e95c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-25T02:48:43.803 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.802+0000 7f5bf81e95c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:43.804 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:43.803+0000 7f5bf81e95c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T02:48:44.047 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.046+0000 7f5bf81e95c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:44.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.046+0000 7f5bf81e95c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:44.048 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.046+0000 7f5bf81e95c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:44.073 DEBUG:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-25T02:48:44.101 DEBUG:teuthology.orchestra.run.smithi101:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:44.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.187+0000 7feec23195c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-25T02:48:44.189 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.187+0000 7feec23195c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-25T02:48:44.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.194+0000 7feec23195c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:44.196 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.194+0000 7feec23195c0 -1 journal FileJournal::_open_file : zeroing journal 2023-01-25T02:48:44.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.389+0000 7feec23195c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:48:44.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.390+0000 7feec23195c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:44.391 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:44.390+0000 7feec23195c0 -1 journal do_read_entry(4096): bad header magic 2023-01-25T02:48:44.398 DEBUG:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-25T02:48:44.426 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T02:48:44.426 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:44.427 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T02:48:44.459 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:44.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T02:48:44.531 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:44.531 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T02:48:44.602 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:44.603 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T02:48:44.630 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:44.631 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-25T02:48:44.697 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:44.698 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-25T02:48:44.765 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:44.765 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-25T02:48:44.831 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:44.832 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T02:48:44.890 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T02:48:44.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T02:48:44.893 DEBUG:teuthology.orchestra.run.smithi101:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T02:48:44.933 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2023-01-25T02:48:44.933 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCHmNBjpFQCMxAA9lpQzHP7s4D2VIpZuakQ7w== 2023-01-25T02:48:44.933 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2023-01-25T02:48:44.934 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCImNBjRcl1EBAA/SZxdMUSi+qLn2+S2C4wfQ== 2023-01-25T02:48:44.934 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2023-01-25T02:48:44.934 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCImNBj4/81KhAAkkcFcSU0+05EqKF4Tkw9EQ== 2023-01-25T02:48:44.934 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.x] 2023-01-25T02:48:44.934 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCFmNBj6+6pIRAA/M53PVxXj24ARfXyihoYLQ== 2023-01-25T02:48:44.935 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.3] 2023-01-25T02:48:44.935 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCLmNBjJWoGFRAA4YSMmxWvdBgXfWIbuGJueg== 2023-01-25T02:48:44.935 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.4] 2023-01-25T02:48:44.935 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCLmNBjPFIyLhAAQLULDFOtfE8RynGO+FLTUw== 2023-01-25T02:48:44.935 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.5] 2023-01-25T02:48:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCMmNBjn8hCCxAANoDXesL4n1Cc0WtHcrmFYQ== 2023-01-25T02:48:44.936 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2023-01-25T02:48:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQCFmNBjLqt5JBAAWZCVyoMToP+7AhtJewZeHw== 2023-01-25T02:48:44.955 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.0] 2023-01-25T02:48:44.956 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCHmNBjpFQCMxAA9lpQzHP7s4D2VIpZuakQ7w== 2023-01-25T02:48:44.956 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.1] 2023-01-25T02:48:44.956 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCImNBjRcl1EBAA/SZxdMUSi+qLn2+S2C4wfQ== 2023-01-25T02:48:44.956 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.2] 2023-01-25T02:48:44.956 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCImNBj4/81KhAAkkcFcSU0+05EqKF4Tkw9EQ== 2023-01-25T02:48:44.957 INFO:teuthology.orchestra.run.smithi101.stdout:[mgr.x] 2023-01-25T02:48:44.957 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCFmNBj6+6pIRAA/M53PVxXj24ARfXyihoYLQ== 2023-01-25T02:48:44.957 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.3] 2023-01-25T02:48:44.957 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCLmNBjJWoGFRAA4YSMmxWvdBgXfWIbuGJueg== 2023-01-25T02:48:44.958 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.4] 2023-01-25T02:48:44.958 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCLmNBjPFIyLhAAQLULDFOtfE8RynGO+FLTUw== 2023-01-25T02:48:44.958 INFO:teuthology.orchestra.run.smithi101.stdout:[osd.5] 2023-01-25T02:48:44.958 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCMmNBjn8hCCxAANoDXesL4n1Cc0WtHcrmFYQ== 2023-01-25T02:48:44.958 INFO:teuthology.orchestra.run.smithi101.stdout:[client.0] 2023-01-25T02:48:44.958 INFO:teuthology.orchestra.run.smithi101.stdout: key = AQCFmNBjLqt5JBAAWZCVyoMToP+7AhtJewZeHw== 2023-01-25T02:48:44.959 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:44.979 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.056 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.112 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.166 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T02:48:45.169 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T02:48:45.222 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.225 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.279 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.281 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.334 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.337 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T02:48:45.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T02:48:45.393 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T02:48:45.449 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T02:48:45.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T02:48:45.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:45.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T02:48:45.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-25T02:48:45.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:45.775 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-25T02:48:45.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-25T02:48:45.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:45.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-25T02:48:45.998 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2023-01-25T02:48:46.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:46.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2023-01-25T02:48:46.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2023-01-25T02:48:46.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:46.362 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2023-01-25T02:48:46.390 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2023-01-25T02:48:46.423 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:46.526 DEBUG:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2023-01-25T02:48:46.555 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2023-01-25T02:48:46.624 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:46.722 DEBUG:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2023-01-25T02:48:46.750 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2023-01-25T02:48:46.817 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:46.915 DEBUG:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2023-01-25T02:48:46.942 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2023-01-25T02:48:47.009 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T02:48:47.106 DEBUG:teuthology.orchestra.run.smithi101:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2023-01-25T02:48:47.133 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:47.136 DEBUG:teuthology.orchestra.run.smithi101:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T02:48:47.190 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T02:48:47.191 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T02:48:47.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T02:48:47.195 INFO:tasks.ceph.mon.a:Started 2023-01-25T02:48:47.195 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-25T02:48:47.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-25T02:48:47.197 INFO:tasks.ceph.mon.b:Started 2023-01-25T02:48:47.198 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-25T02:48:47.198 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-25T02:48:47.200 INFO:tasks.ceph.mon.c:Started 2023-01-25T02:48:47.201 INFO:tasks.ceph.mon.d:Restarting daemon 2023-01-25T02:48:47.201 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2023-01-25T02:48:47.203 INFO:tasks.ceph.mon.d:Started 2023-01-25T02:48:47.203 INFO:tasks.ceph.mon.e:Restarting daemon 2023-01-25T02:48:47.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2023-01-25T02:48:47.206 INFO:tasks.ceph.mon.e:Started 2023-01-25T02:48:47.206 INFO:tasks.ceph.mon.f:Restarting daemon 2023-01-25T02:48:47.206 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2023-01-25T02:48:47.233 INFO:tasks.ceph.mon.f:Started 2023-01-25T02:48:47.233 INFO:tasks.ceph.mon.g:Restarting daemon 2023-01-25T02:48:47.233 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2023-01-25T02:48:47.235 INFO:tasks.ceph.mon.g:Started 2023-01-25T02:48:47.236 INFO:tasks.ceph.mon.h:Restarting daemon 2023-01-25T02:48:47.236 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2023-01-25T02:48:47.238 INFO:tasks.ceph.mon.h:Started 2023-01-25T02:48:47.238 INFO:tasks.ceph.mon.i:Restarting daemon 2023-01-25T02:48:47.239 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2023-01-25T02:48:47.242 INFO:tasks.ceph.mon.i:Started 2023-01-25T02:48:47.243 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T02:48:47.243 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T02:48:47.243 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T02:48:47.245 INFO:tasks.ceph.mgr.x:Started 2023-01-25T02:48:47.246 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T02:48:47.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T02:48:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.404+0000 7f3aafb1c700 10 monclient: get_monmap_and_config 2023-01-25T02:48:47.405 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.404+0000 7f3aafb1c700 10 monclient: build_initial_monmap 2023-01-25T02:48:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.404+0000 7f3aafb1c700 10 monclient: monmap: 2023-01-25T02:48:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-01-25T02:48:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:47.405407+0000 2023-01-25T02:48:47.406 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:47.405407+0000 2023-01-25T02:48:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-01-25T02:48:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:48:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:48:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:48:47.407 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:48:47.408 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:48:47.408 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:48:47.408 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:48:47.408 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:48:47.408 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:48:47.409 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:48:47.409 INFO:teuthology.orchestra.run.smithi019.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:48:47.409 INFO:teuthology.orchestra.run.smithi019.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:48:47.409 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:48:47.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 1 Processor -- start 2023-01-25T02:48:47.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 1 -- start start 2023-01-25T02:48:47.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient: init 2023-01-25T02:48:47.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:47.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient: _add_conns ranks=[6,9,8,1,7,0,5,4,2,10,3] 2023-01-25T02:48:47.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:47.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient(hunting): picked mon.noname-d con 0x7f3aa80a3350 addr [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] 2023-01-25T02:48:47.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:47.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient(hunting): picked mon.noname-a con 0x7f3aa80a3720 addr v2:172.21.15.19:3300/0 2023-01-25T02:48:47.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:47.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient(hunting): picked mon.noname-e con 0x7f3aa8136190 addr [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] 2023-01-25T02:48:47.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:47.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa80a1390 con 0x7f3aa80a3720 2023-01-25T02:48:47.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:47.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 1 -- --> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80a1600 con 0x7f3aa80a3350 2023-01-25T02:48:47.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:47.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 1 -- --> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80a18e0 con 0x7f3aa8136190 2023-01-25T02:48:47.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:47.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.405+0000 7f3aafb1c700 10 monclient(hunting): authenticate will time out at 936.930942s 2023-01-25T02:48:47.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:47.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:47.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:47.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3304/0 says I am v2:172.21.15.19:48888/0 (socket says 172.21.15.19:48888) 2023-01-25T02:48:47.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58954/0 (socket says 172.21.15.19:58954) 2023-01-25T02:48:47.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 learned_addr learned my addr 172.21.15.19:0/1449225867 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-01-25T02:48:47.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request con 0x7f3aa8136190 auth_method 0 2023-01-25T02:48:47.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:47.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:47.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:47.417 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3aa80a3720 auth_method 0 2023-01-25T02:48:47.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:47.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:47.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:47.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3aa80a3350 auth_method 0 2023-01-25T02:48:47.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:47.418 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:47.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:47.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-01-25T02:48:47.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:47.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:47.419 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:47.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-01-25T02:48:47.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:47.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:47.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:47.420 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:47.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T02:48:47.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T02:48:47.421 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.406+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T02:48:47.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:47.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request con 0x7f3aa8136190 auth_method 0 2023-01-25T02:48:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:47.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3aa80a3720 auth_method 0 2023-01-25T02:48:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3aa80a3350 auth_method 0 2023-01-25T02:48:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:47.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-01-25T02:48:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:47.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-01-25T02:48:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T02:48:47.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T02:48:47.614 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:47.607+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T02:48:48.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:48.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:48.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:48.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3aa80a3720 auth_method 0 2023-01-25T02:48:48.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:48.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:48.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:48.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3aa80a3350 auth_method 0 2023-01-25T02:48:48.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:48.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:48.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:48.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request con 0x7f3aa8136190 auth_method 0 2023-01-25T02:48:48.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:48.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:48.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:48.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-01-25T02:48:48.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:48.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:48.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-01-25T02:48:48.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:48.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:48.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:48.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:48.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:48.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-01-25T02:48:48.014 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-01-25T02:48:48.015 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.008+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-01-25T02:48:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:48.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request con 0x7f3aa8136190 auth_method 0 2023-01-25T02:48:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:48.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3aa80a3350 auth_method 0 2023-01-25T02:48:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:48.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3aa80a3720 auth_method 0 2023-01-25T02:48:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:48.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-01-25T02:48:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-01-25T02:48:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:48.814 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:48.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:48.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:48.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-01-25T02:48:48.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-01-25T02:48:48.815 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:48.809+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-01-25T02:48:50.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): tick 2023-01-25T02:48:50.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): _check_auth_tickets 2023-01-25T02:48:50.408 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 monclient(hunting): continuing hunt 2023-01-25T02:48:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): _reopen_session rank -1 2023-01-25T02:48:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 msgr2=0x7f3aa8136600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-01-25T02:48:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3aa8136190 0x7f3aa8136600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 msgr2=0x7f3aa80a22c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-01-25T02:48:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3aa80a3350 0x7f3aa80a22c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 msgr2=0x7f3aa80a3b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-01-25T02:48:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.19:3300/0 conn(0x7f3aa80a3720 0x7f3aa80a3b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient: _add_conns ranks=[4,10,1,2,0,3,5,7] 2023-01-25T02:48:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): picked mon.noname-c con 0x7f3a940038b0 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:48:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): picked mon.noname-f con 0x7f3a940062b0 addr v2:172.21.15.101:3300/0 2023-01-25T02:48:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f3a94008df0 addr v1:172.21.15.101:6789/0 2023-01-25T02:48:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 -- 172.21.15.19:0/1449225867 --> v1:172.21.15.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa8061b00 con 0x7f3a94008df0 2023-01-25T02:48:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 -- 172.21.15.19:0/1449225867 --> v2:172.21.15.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa80a1390 con 0x7f3a940062b0 2023-01-25T02:48:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 1 -- 172.21.15.19:0/1449225867 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80a1600 con 0x7f3a940038b0 2023-01-25T02:48:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3a9f7fe700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3a940038b0 auth_method 0 2023-01-25T02:48:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:50.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:50.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T02:48:50.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:50.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3a940062b0 auth_method 0 2023-01-25T02:48:50.414 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-01-25T02:48:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.406+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T02:48:50.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:50.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3a940038b0 auth_method 0 2023-01-25T02:48:50.608 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:50.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:50.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:50.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:50.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:50.609 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:50.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:50.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T02:48:50.610 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3a940062b0 auth_method 0 2023-01-25T02:48:50.611 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:50.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:50.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:50.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-01-25T02:48:50.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:50.612 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:50.613 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:50.607+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T02:48:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3a940038b0 auth_method 0 2023-01-25T02:48:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-01-25T02:48:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-01-25T02:48:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3a940062b0 auth_method 0 2023-01-25T02:48:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.008+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-01-25T02:48:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:51.810 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3a940038b0 auth_method 0 2023-01-25T02:48:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-01-25T02:48:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3a940062b0 auth_method 0 2023-01-25T02:48:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:51.812 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-01-25T02:48:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:51.813 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:51.814 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:51.809+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-01-25T02:48:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 <== mon.1 v1:172.21.15.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028115696 0 0) 0x7f3aa8061b00 con 0x7f3a94008df0 2023-01-25T02:48:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 10 monclient(hunting): _negotiate 2023-01-25T02:48:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 10 monclient(hunting): my global_id is 4105 2023-01-25T02:48:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 --> v1:172.21.15.101:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a8c0035b0 con 0x7f3a94008df0 2023-01-25T02:48:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 <== mon.1 v1:172.21.15.101:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643466087 0 0) 0x7f3a8c0035b0 con 0x7f3a94008df0 2023-01-25T02:48:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 msgr2=0x7f3a94005cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-01-25T02:48:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a940038b0 0x7f3a94005cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 msgr2=0x7f3a94008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-01-25T02:48:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 --2- 172.21.15.19:0/1449225867 >> v2:172.21.15.101:3300/0 conn(0x7f3a940062b0 0x7f3a94008730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 10 monclient: _finish_hunting 0 2023-01-25T02:48:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 monclient: found mon.noname-f-legacy 2023-01-25T02:48:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aae0b9700 1 -- 172.21.15.19:0/1449225867 reap_dead start 2023-01-25T02:48:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 10 monclient: _send_mon_message to mon.noname-f-legacy at v1:172.21.15.101:6789/0 2023-01-25T02:48:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 --> v1:172.21.15.101:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9400dc00 con 0x7f3a94008df0 2023-01-25T02:48:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 10 monclient: _finish_auth 0 2023-01-25T02:48:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 10 monclient: _check_auth_tickets 2023-01-25T02:48:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aafb1c700 5 monclient: authenticate success, global_id 4105 2023-01-25T02:48:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aafb1c700 20 monclient: get_monmap_and_config waiting for monmap|config 2023-01-25T02:48:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.628+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 <== mon.1 v1:172.21.15.101:6789/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (unknown 998048899 0 0) 0x7f3aa0009240 con 0x7f3a94008df0 2023-01-25T02:48:52.642 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:52.642 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient: got monmap 1 from mon.noname-f-legacy (according to old e1) 2023-01-25T02:48:52.642 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient: dump: 2023-01-25T02:48:52.642 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:48:52.642 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:48:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:52.644 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:52.644 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:52.644 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:52.644 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:52.644 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:52.645 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:48:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 1 monclient: mon.f has (v2) addrs [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] but i'm connected to v1:172.21.15.101:6789/0, reconnecting 2023-01-25T02:48:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 >> v1:172.21.15.101:6789/0 conn(0x7f3a94008df0 legacy=0x7f3a9400b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient: _add_conns ranks=[3,2,7,5,1,4,0,6,8] 2023-01-25T02:48:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f3a8c004250 0x7f3a8c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient(hunting): picked mon.g con 0x7f3a8c004250 addr [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] 2023-01-25T02:48:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3a8c005320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient(hunting): picked mon.b con 0x7f3a8c004eb0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-01-25T02:48:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3a8c009c40 0x7f3a8c00c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient(hunting): picked mon.i con 0x7f3a8c009c40 addr [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] 2023-01-25T02:48:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80a1390 con 0x7f3a8c004eb0 2023-01-25T02:48:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80a1600 con 0x7f3a8c004250 2023-01-25T02:48:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c00c9b0 con 0x7f3a8c009c40 2023-01-25T02:48:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aac8b6700 10 monclient(hunting): _finish_auth 0 2023-01-25T02:48:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aafb1c700 20 monclient(hunting): get_monmap_and_config waiting for monmap|config 2023-01-25T02:48:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3a8c005320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3a8c004eb0 auth_method 0 2023-01-25T02:48:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f3a8c004250 0x7f3a8c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3a8c009c40 0x7f3a8c00c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:52.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request con 0x7f3a8c009c40 auth_method 0 2023-01-25T02:48:52.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:52.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aae0b9700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:52.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aae0b9700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:52.651 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3a8c004250 auth_method 0 2023-01-25T02:48:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 10 monclient(hunting): _init_auth already have auth, reseting 2023-01-25T02:48:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-01-25T02:48:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-01-25T02:48:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aae0b9700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aae0b9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aae0b9700 10 monclient(hunting): handle_auth_reply_more responding with 132 bytes 2023-01-25T02:48:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient(hunting): handle_auth_done global_id 4105 payload 293 2023-01-25T02:48:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3a8c009c40 msgr2=0x7f3a8c00c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:48:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3a8c009c40 0x7f3a8c00c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f3a8c004250 msgr2=0x7f3a8c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:48:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f3a8c004250 0x7f3a8c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient: _finish_hunting 0 2023-01-25T02:48:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 1 monclient: found mon.b 2023-01-25T02:48:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.19:3301/0 2023-01-25T02:48:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 1 -- 172.21.15.19:0/1449225867 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0035b0 con 0x7f3a8c004eb0 2023-01-25T02:48:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient: _finish_auth 0 2023-01-25T02:48:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 10 monclient: _check_auth_tickets 2023-01-25T02:48:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f3a8c004250 0x7f3a8c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:48:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.629+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3a8c005320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a98019ad0 tx=0x7f3a8c005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb79cef2e61feeeb server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:48:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a9801e070 con 0x7f3a8c004eb0 2023-01-25T02:48:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-01-25T02:48:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 10 monclient: dump: 2023-01-25T02:48:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:48:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:48:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:52.661 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:48:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9800c040 con 0x7f3a8c004eb0 2023-01-25T02:48:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 20 monclient: get_monmap_and_config waiting for monmap|config 2023-01-25T02:48:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:48:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 10 monclient: get_monmap_and_config success 2023-01-25T02:48:52.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 1 -- 172.21.15.19:0/1449225867 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a98016d30 con 0x7f3a8c004eb0 2023-01-25T02:48:52.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:52.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 4 set_mon_vals no callback set 2023-01-25T02:48:52.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-01-25T02:48:52.662 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aac8b6700 10 monclient: dump: 2023-01-25T02:48:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:48:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:48:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:52.665 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:48:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 10 monclient: shutdown 2023-01-25T02:48:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:48:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 1 -- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 msgr2=0x7f3a8c005320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3a8c005320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3a98019ad0 tx=0x7f3a8c005880 comp rx=0 tx=0).stop 2023-01-25T02:48:52.666 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 1 -- 172.21.15.19:0/1449225867 shutdown_connections 2023-01-25T02:48:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3a8c005320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f3a8c004250 0x7f3a8c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 1 --2- 172.21.15.19:0/1449225867 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3a8c009c40 0x7f3a8c00c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.630+0000 7f3aafb1c700 1 -- 172.21.15.19:0/1449225867 >> 172.21.15.19:0/1449225867 conn(0x7f3aa806c080 msgr2=0x7f3aa806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:48:52.667 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.631+0000 7f3aafb1c700 1 -- 172.21.15.19:0/1449225867 shutdown_connections 2023-01-25T02:48:52.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.631+0000 7f3aafb1c700 1 -- 172.21.15.19:0/1449225867 wait complete. 2023-01-25T02:48:52.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.631+0000 7f3aafb1c700 10 monclient: build_initial_monmap 2023-01-25T02:48:52.668 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.631+0000 7f3aafb1c700 10 monclient: monmap: 2023-01-25T02:48:52.668 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-01-25T02:48:52.668 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:48:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-01-25T02:48:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-01-25T02:48:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-01-25T02:48:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-01-25T02:48:52.669 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:48:52.670 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:48:52.670 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:48:52.670 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:48:52.670 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:48:52.670 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:48:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:48:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:48:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:48:52.671 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:48:52.671 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 1 Processor -- start 2023-01-25T02:48:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 1 -- start start 2023-01-25T02:48:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient: init 2023-01-25T02:48:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient: _add_conns ranks=[6,2,4,0,1,7,5,8,3] 2023-01-25T02:48:52.672 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3a8c004250 0x7f3aa81c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:52.673 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient(hunting): picked mon.noname-d con 0x7f3a8c004250 addr [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] 2023-01-25T02:48:52.673 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3aa81355e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:52.673 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient(hunting): picked mon.noname-b con 0x7f3a8c004eb0 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-01-25T02:48:52.673 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a8c009c40 0x7f3aa8135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:52.673 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient(hunting): picked mon.noname-c con 0x7f3a8c009c40 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:48:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa809b9e0 con 0x7f3a8c004eb0 2023-01-25T02:48:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa8121a30 con 0x7f3a8c009c40 2023-01-25T02:48:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:52.674 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 1 -- --> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80a0fd0 con 0x7f3a8c004250 2023-01-25T02:48:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aafb1c700 10 monclient(hunting): authenticate will time out at 942.157873s 2023-01-25T02:48:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aad8b8700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3a8c004250 0x7f3aa81c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.632+0000 7f3aad0b7700 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3aa81355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3aa81355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3301/0 says I am v2:172.21.15.19:47308/0 (socket says 172.21.15.19:47308) 2023-01-25T02:48:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3a8c004250 0x7f3aa81c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3303/0 says I am v2:172.21.15.19:46176/0 (socket says 172.21.15.19:46176) 2023-01-25T02:48:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 -- 172.21.15.19:0/3509588372 learned_addr learned my addr 172.21.15.19:0/3509588372 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-01-25T02:48:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request con 0x7f3a8c004eb0 auth_method 0 2023-01-25T02:48:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request con 0x7f3a8c004250 auth_method 0 2023-01-25T02:48:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aae0b9700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a8c009c40 0x7f3aa8135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request con 0x7f3a8c009c40 auth_method 0 2023-01-25T02:48:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aae0b9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aae0b9700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aae0b9700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aae0b9700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aae0b9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aae0b9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient(hunting): handle_auth_done global_id 4106 payload 824 2023-01-25T02:48:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 -- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3a8c004250 msgr2=0x7f3aa81c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3a8c004250 0x7f3aa81c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 -- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a8c009c40 msgr2=0x7f3aa8135b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a8c009c40 0x7f3aa8135b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient: _finish_hunting 0 2023-01-25T02:48:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 monclient: found mon.noname-b 2023-01-25T02:48:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 10 monclient: _send_mon_message to mon.noname-b at v2:172.21.15.19:3301/0 2023-01-25T02:48:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 -- 172.21.15.19:0/3509588372 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80a0c20 con 0x7f3a8c004eb0 2023-01-25T02:48:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad8b8700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3a8c004250 0x7f3aa81c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:48:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aad0b7700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3aa81355e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a98019aa0 tx=0x7f3a98004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:48:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 1 -- 172.21.15.19:0/3509588372 <== mon.2 v2:172.21.15.19:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a9802cac0 con 0x7f3a8c004eb0 2023-01-25T02:48:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2023-01-25T02:48:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: dump: 2023-01-25T02:48:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:48:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.684 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:48:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:52.686 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:48:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: _finish_auth 0 2023-01-25T02:48:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: _check_auth_tickets 2023-01-25T02:48:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 1 -- 172.21.15.19:0/3509588372 <== mon.2 v2:172.21.15.19:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9802cc60 con 0x7f3a8c004eb0 2023-01-25T02:48:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aafb1c700 5 monclient: authenticate success, global_id 4106 2023-01-25T02:48:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aafb1c700 10 monclient: _renew_subs 2023-01-25T02:48:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aafb1c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.19:3301/0 2023-01-25T02:48:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa80a0000 con 0x7f3a8c004eb0 2023-01-25T02:48:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:48:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 1 -- 172.21.15.19:0/3509588372 <== mon.2 v2:172.21.15.19:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3a9802cdd0 con 0x7f3a8c004eb0 2023-01-25T02:48:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: got monmap 1 from mon.b (according to old e1) 2023-01-25T02:48:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9e7fc700 10 monclient: dump: 2023-01-25T02:48:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:48:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:48:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:52.692 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:52.692 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:52.692 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:52.692 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:48:52.692 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3a9ffff700 4 set_mon_vals no callback set 2023-01-25T02:48:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aafb1c700 10 monclient: _renew_subs 2023-01-25T02:48:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aafb1c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.19:3301/0 2023-01-25T02:48:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.633+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa80a3b50 con 0x7f3a8c004eb0 2023-01-25T02:48:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.634+0000 7f3a9e7fc700 1 -- 172.21.15.19:0/3509588372 <== mon.2 v2:172.21.15.19:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3a98012040 con 0x7f3a8c004eb0 2023-01-25T02:48:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.634+0000 7f3a9e7fc700 1 -- 172.21.15.19:0/3509588372 <== mon.2 v2:172.21.15.19:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3a98022910 con 0x7f3a8c004eb0 2023-01-25T02:48:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.635+0000 7f3aafb1c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.635+0000 7f3aafb1c700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.635+0000 7f3aafb1c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.19:3301/0 2023-01-25T02:48:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.635+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa806c9b0 con 0x7f3a8c004eb0 2023-01-25T02:48:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.637+0000 7f3a9e7fc700 1 -- 172.21.15.19:0/3509588372 <== mon.2 v2:172.21.15.19:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f3a98053860 con 0x7f3a8c004eb0 2023-01-25T02:48:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.637+0000 7f3a9e7fc700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.637+0000 7f3a9e7fc700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:48:52.714 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.714+0000 7f3aafb1c700 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2023-01-25T02:48:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.714+0000 7f3aafb1c700 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2023-01-25T02:48:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.714+0000 7f3aafb1c700 10 monclient: _send_mon_message to mon.b at v2:172.21.15.19:3301/0 2023-01-25T02:48:52.715 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.714+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3aa8066690 con 0x7f3a8c004eb0 2023-01-25T02:48:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.717+0000 7f3a9e7fc700 1 -- 172.21.15.19:0/3509588372 <== mon.2 v2:172.21.15.19:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3a98042070 con 0x7f3a8c004eb0 2023-01-25T02:48:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.717+0000 7f3a9e7fc700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2023-01-25T02:48:52.718 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.717+0000 7f3a9e7fc700 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2023-01-25T02:48:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 10 monclient: shutdown 2023-01-25T02:48:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:48:52.719 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 msgr2=0x7f3aa81355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3aa81355e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3a98019aa0 tx=0x7f3a98004750 comp rx=0 tx=0).stop 2023-01-25T02:48:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 shutdown_connections 2023-01-25T02:48:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f3a8c009c40 0x7f3aa8135b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3a8c004250 0x7f3aa81c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.720 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 --2- 172.21.15.19:0/3509588372 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f3a8c004eb0 0x7f3aa81355e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 >> 172.21.15.19:0/3509588372 conn(0x7f3aa806c080 msgr2=0x7f3aa80997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:48:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 shutdown_connections 2023-01-25T02:48:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:48:52.718+0000 7f3aafb1c700 1 -- 172.21.15.19:0/3509588372 wait complete. 2023-01-25T02:48:52.721 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2023-01-25T02:48:52.731 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T02:48:52.731 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:52.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T02:48:52.760 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:52.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T02:48:52.828 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2023-01-25T02:48:52.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T02:48:52.896 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:52.896 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-25T02:48:52.929 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:52.930 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-25T02:48:53.000 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-01-25T02:48:53.001 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-25T02:48:53.070 DEBUG:teuthology.orchestra.run.smithi101:> sudo ceph --cluster ceph osd new 635bff03-56e6-4b89-821e-97ea356fb68c 0 2023-01-25T02:48:53.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.229+0000 7eff8870f700 10 monclient: get_monmap_and_config 2023-01-25T02:48:53.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.229+0000 7eff8870f700 10 monclient: build_initial_monmap 2023-01-25T02:48:53.229 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.229+0000 7eff8870f700 10 monclient: monmap: 2023-01-25T02:48:53.230 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:48:53.230 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.230 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:53.229649+0000 2023-01-25T02:48:53.230 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:53.229649+0000 2023-01-25T02:48:53.230 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:48:53.231 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.231 INFO:teuthology.orchestra.run.smithi101.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:48:53.231 INFO:teuthology.orchestra.run.smithi101.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:48:53.231 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:48:53.231 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:48:53.232 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:48:53.232 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:48:53.232 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:48:53.232 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:48:53.232 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:48:53.233 INFO:teuthology.orchestra.run.smithi101.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:48:53.233 INFO:teuthology.orchestra.run.smithi101.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:48:53.233 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.241 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.229+0000 7eff8870f700 1 Processor -- start 2023-01-25T02:48:53.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.229+0000 7eff8870f700 1 -- start start 2023-01-25T02:48:53.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.229+0000 7eff8870f700 10 monclient: init 2023-01-25T02:48:53.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:53.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient: _add_conns ranks=[5,4,10,8,2,6,0,7,9,3,1] 2023-01-25T02:48:53.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff800a35b0 0x7eff800a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient(hunting): picked mon.noname-h con 0x7eff800a35b0 addr [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] 2023-01-25T02:48:53.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 0x7eff800a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient(hunting): picked mon.noname-c con 0x7eff800a4f50 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:48:53.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 1 --2- >> v2:172.21.15.101:3300/0 conn(0x7eff80135fa0 0x7eff80136440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient(hunting): picked mon.noname-f con 0x7eff80135fa0 addr v2:172.21.15.101:3300/0 2023-01-25T02:48:53.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 1 -- --> v2:172.21.15.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff800a5540 con 0x7eff80135fa0 2023-01-25T02:48:53.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff800a4400 con 0x7eff800a4f50 2023-01-25T02:48:53.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 1 -- --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff8006f7b0 con 0x7eff800a35b0 2023-01-25T02:48:53.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:53.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff8870f700 10 monclient(hunting): authenticate will time out at 938.853629s 2023-01-25T02:48:53.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 1 --2- >> v2:172.21.15.101:3300/0 conn(0x7eff80135fa0 0x7eff80136440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff864ab700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff800a35b0 0x7eff800a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 1 --2- >> v2:172.21.15.101:3300/0 conn(0x7eff80135fa0 0x7eff80136440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:48542/0 (socket says 172.21.15.101:48542) 2023-01-25T02:48:53.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 1 -- 172.21.15.101:0/184680305 learned_addr learned my addr 172.21.15.101:0/184680305 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:48:53.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 10 monclient(hunting): get_auth_request con 0x7eff80135fa0 auth_method 0 2023-01-25T02:48:53.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.246 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff864ab700 10 monclient(hunting): get_auth_request con 0x7eff800a35b0 auth_method 0 2023-01-25T02:48:53.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff864ab700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff864ab700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff864ab700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.247 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff86cac700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff864ab700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff864ab700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff864ab700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.230+0000 7eff85caa700 1 --2- 172.21.15.101:0/184680305 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 0x7eff800a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 10 monclient(hunting): handle_auth_done global_id 4109 payload 274 2023-01-25T02:48:53.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 1 -- 172.21.15.101:0/184680305 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 msgr2=0x7eff800a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:48:53.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 1 --2- 172.21.15.101:0/184680305 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 0x7eff800a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 1 -- 172.21.15.101:0/184680305 >> v2:172.21.15.101:3300/0 conn(0x7eff80135fa0 msgr2=0x7eff80136440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:48:53.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 1 --2- 172.21.15.101:0/184680305 >> v2:172.21.15.101:3300/0 conn(0x7eff80135fa0 0x7eff80136440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 10 monclient: _finish_hunting 0 2023-01-25T02:48:53.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 1 monclient: found mon.noname-h 2023-01-25T02:48:53.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:53.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.101:3302/0 2023-01-25T02:48:53.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 1 -- 172.21.15.101:0/184680305 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80061b00 con 0x7eff800a35b0 2023-01-25T02:48:53.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff86cac700 1 --2- 172.21.15.101:0/184680305 >> v2:172.21.15.101:3300/0 conn(0x7eff80135fa0 0x7eff80136440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:48:53.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff864ab700 1 --2- 172.21.15.101:0/184680305 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff800a35b0 0x7eff800a3980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7eff70009840 tx=0x7eff70009b50 comp rx=0 tx=0).ready entity=mon.5 client_cookie=c7bce2aae49a63cd server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:48:53.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 1 -- 172.21.15.101:0/184680305 <== mon.5 v2:172.21.15.101:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff70011aa0 con 0x7eff800a35b0 2023-01-25T02:48:53.251 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2023-01-25T02:48:53.252 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: dump: 2023-01-25T02:48:53.252 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.252 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.252 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.252 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.253 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.253 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.253 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.253 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.253 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.254 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.254 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.254 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.254 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.254 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.255 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.255 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: _finish_auth 0 2023-01-25T02:48:53.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: _check_auth_tickets 2023-01-25T02:48:53.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:53.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 1 -- 172.21.15.101:0/184680305 <== mon.5 v2:172.21.15.101:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70011c00 con 0x7eff800a35b0 2023-01-25T02:48:53.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 5 monclient: authenticate success, global_id 4109 2023-01-25T02:48:53.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 20 monclient: get_monmap_and_config waiting for monmap|config 2023-01-25T02:48:53.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:48:53.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 1 -- 172.21.15.101:0/184680305 <== mon.5 v2:172.21.15.101:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff70011d70 con 0x7eff800a35b0 2023-01-25T02:48:53.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 10 monclient: get_monmap_and_config success 2023-01-25T02:48:53.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 4 set_mon_vals no callback set 2023-01-25T02:48:53.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: got monmap 1 from mon.h (according to old e1) 2023-01-25T02:48:53.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff854a9700 10 monclient: dump: 2023-01-25T02:48:53.258 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.258 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.258 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.258 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.258 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.259 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.259 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.259 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.259 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.259 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.259 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.260 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.260 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.260 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.260 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.260 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 10 monclient: shutdown 2023-01-25T02:48:53.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:48:53.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 1 -- 172.21.15.101:0/184680305 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff800a35b0 msgr2=0x7eff800a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 1 --2- 172.21.15.101:0/184680305 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff800a35b0 0x7eff800a3980 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7eff70009840 tx=0x7eff70009b50 comp rx=0 tx=0).stop 2023-01-25T02:48:53.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 1 -- 172.21.15.101:0/184680305 shutdown_connections 2023-01-25T02:48:53.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 1 --2- 172.21.15.101:0/184680305 >> v2:172.21.15.101:3300/0 conn(0x7eff80135fa0 0x7eff80136440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 1 --2- 172.21.15.101:0/184680305 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff800a35b0 0x7eff800a3980 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 1 --2- 172.21.15.101:0/184680305 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 0x7eff800a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.231+0000 7eff8870f700 1 -- 172.21.15.101:0/184680305 >> 172.21.15.101:0/184680305 conn(0x7eff800a57b0 msgr2=0x7eff800a5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:48:53.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.232+0000 7eff8870f700 1 -- 172.21.15.101:0/184680305 shutdown_connections 2023-01-25T02:48:53.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.232+0000 7eff8870f700 1 -- 172.21.15.101:0/184680305 wait complete. 2023-01-25T02:48:53.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.232+0000 7eff8870f700 10 monclient: build_initial_monmap 2023-01-25T02:48:53.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.232+0000 7eff8870f700 10 monclient: monmap: 2023-01-25T02:48:53.263 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:48:53.263 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:48:53.263 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 0.000000 2023-01-25T02:48:53.264 INFO:teuthology.orchestra.run.smithi101.stderr:created 0.000000 2023-01-25T02:48:53.264 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:48:53.264 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 1 2023-01-25T02:48:53.264 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:48:53.264 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:48:53.265 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:48:53.265 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:48:53.265 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:48:53.265 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:48:53.265 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:48:53.266 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:48:53.266 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:48:53.266 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.232+0000 7eff8870f700 1 Processor -- start 2023-01-25T02:48:53.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.232+0000 7eff8870f700 1 -- start start 2023-01-25T02:48:53.266 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.232+0000 7eff8870f700 10 monclient: init 2023-01-25T02:48:53.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:53.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient: _add_conns ranks=[7,4,5,8,0,2,6,1,3] 2023-01-25T02:48:53.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7eff800a35b0 0x7eff801ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient(hunting): picked mon.noname-i con 0x7eff800a35b0 addr [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] 2023-01-25T02:48:53.267 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 0x7eff801caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient(hunting): picked mon.noname-c con 0x7eff800a4f50 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:48:53.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff80135fa0 0x7eff8011f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient(hunting): picked mon.noname-h con 0x7eff80135fa0 addr [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] 2023-01-25T02:48:53.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff801232e0 con 0x7eff800a4f50 2023-01-25T02:48:53.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 1 -- --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff80123530 con 0x7eff80135fa0 2023-01-25T02:48:53.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 1 -- --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_getmap magic: 0 v1 -- 0x7eff80121da0 con 0x7eff800a35b0 2023-01-25T02:48:53.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:53.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff8870f700 10 monclient(hunting): authenticate will time out at 938.856401s 2023-01-25T02:48:53.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff80135fa0 0x7eff8011f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7eff800a35b0 0x7eff801ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7eff800a35b0 0x7eff801ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3303/0 says I am v2:172.21.15.101:47164/0 (socket says 172.21.15.101:47164) 2023-01-25T02:48:53.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient(hunting): get_auth_request con 0x7eff80135fa0 auth_method 0 2023-01-25T02:48:53.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 1 -- 172.21.15.101:0/3831384635 learned_addr learned my addr 172.21.15.101:0/3831384635 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:48:53.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 10 monclient(hunting): get_auth_request con 0x7eff800a35b0 auth_method 0 2023-01-25T02:48:53.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff85caa700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 0x7eff801caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.272 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.273 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff85caa700 10 monclient(hunting): get_auth_request con 0x7eff800a4f50 auth_method 0 2023-01-25T02:48:53.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff85caa700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff85caa700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff85caa700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff85caa700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff85caa700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff85caa700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient(hunting): handle_auth_done global_id 4118 payload 824 2023-01-25T02:48:53.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 1 -- 172.21.15.101:0/3831384635 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7eff800a35b0 msgr2=0x7eff801ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7eff800a35b0 0x7eff801ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 1 -- 172.21.15.101:0/3831384635 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 msgr2=0x7eff801caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 0x7eff801caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient: _finish_hunting 0 2023-01-25T02:48:53.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 1 monclient: found mon.noname-h 2023-01-25T02:48:53.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:53.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.101:3302/0 2023-01-25T02:48:53.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 1 -- 172.21.15.101:0/3831384635 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78009b30 con 0x7eff80135fa0 2023-01-25T02:48:53.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff864ab700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7eff800a35b0 0x7eff801ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:48:53.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.233+0000 7eff86cac700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff80135fa0 0x7eff8011f4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff78009db0 tx=0x7eff7800c3a0 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:48:53.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 1 -- 172.21.15.101:0/3831384635 <== mon.5 v2:172.21.15.101:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff78018070 con 0x7eff80135fa0 2023-01-25T02:48:53.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.277 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2023-01-25T02:48:53.278 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: dump: 2023-01-25T02:48:53.278 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.278 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.278 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.278 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.279 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.279 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.279 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.279 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.279 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.279 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.280 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.280 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.280 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.280 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.280 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.281 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: _finish_auth 0 2023-01-25T02:48:53.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: _check_auth_tickets 2023-01-25T02:48:53.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:53.281 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff8870f700 5 monclient: authenticate success, global_id 4118 2023-01-25T02:48:53.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 1 -- 172.21.15.101:0/3831384635 <== mon.5 v2:172.21.15.101:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7800f040 con 0x7eff80135fa0 2023-01-25T02:48:53.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:48:53.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 1 -- 172.21.15.101:0/3831384635 <== mon.5 v2:172.21.15.101:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7eff7800ccf0 con 0x7eff80135fa0 2023-01-25T02:48:53.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.282 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: got monmap 1 from mon.h (according to old e1) 2023-01-25T02:48:53.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff84ca8700 4 set_mon_vals no callback set 2023-01-25T02:48:53.283 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 10 monclient: dump: 2023-01-25T02:48:53.283 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.283 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.283 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.283 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.284 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.284 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.284 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.284 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.284 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.285 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.285 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.285 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.285 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.285 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.285 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.286 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff8870f700 10 monclient: _renew_subs 2023-01-25T02:48:53.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff8870f700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.101:3302/0 2023-01-25T02:48:53.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff700094a0 con 0x7eff80135fa0 2023-01-25T02:48:53.286 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff8870f700 10 monclient: _renew_subs 2023-01-25T02:48:53.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff8870f700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.101:3302/0 2023-01-25T02:48:53.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8006fd70 con 0x7eff80135fa0 2023-01-25T02:48:53.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 1 -- 172.21.15.101:0/3831384635 <== mon.5 v2:172.21.15.101:3302/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7eff7800c880 con 0x7eff80135fa0 2023-01-25T02:48:53.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.234+0000 7eff76ffd700 1 -- 172.21.15.101:0/3831384635 <== mon.5 v2:172.21.15.101:3302/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7eff780043a0 con 0x7eff80135fa0 2023-01-25T02:48:53.287 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.235+0000 7eff8870f700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.235+0000 7eff8870f700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.235+0000 7eff8870f700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.101:3302/0 2023-01-25T02:48:53.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.235+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8006fa90 con 0x7eff80135fa0 2023-01-25T02:48:53.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.237+0000 7eff76ffd700 1 -- 172.21.15.101:0/3831384635 <== mon.5 v2:172.21.15.101:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7eff78003a20 con 0x7eff80135fa0 2023-01-25T02:48:53.288 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.237+0000 7eff76ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.289 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.237+0000 7eff76ffd700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:48:53.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.317+0000 7eff8870f700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "635bff03-56e6-4b89-821e-97ea356fb68c", "id": 0}] 2023-01-25T02:48:53.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.317+0000 7eff8870f700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "635bff03-56e6-4b89-821e-97ea356fb68c", "id": 0}] 2023-01-25T02:48:53.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.317+0000 7eff8870f700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.101:3302/0 2023-01-25T02:48:53.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.317+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_command({"prefix": "osd new", "uuid": "635bff03-56e6-4b89-821e-97ea356fb68c", "id": 0} v 0) v1 -- 0x7eff8006f140 con 0x7eff80135fa0 2023-01-25T02:48:53.325 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.325+0000 7eff76ffd700 1 -- 172.21.15.101:0/3831384635 <== mon.5 v2:172.21.15.101:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "635bff03-56e6-4b89-821e-97ea356fb68c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7eff78004560 con 0x7eff80135fa0 2023-01-25T02:48:53.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.325+0000 7eff76ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "635bff03-56e6-4b89-821e-97ea356fb68c", "id": 0}] 2023-01-25T02:48:53.326 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.325+0000 7eff76ffd700 10 monclient: _finish_command 2 = system:0 2023-01-25T02:48:53.326 INFO:teuthology.orchestra.run.smithi101.stdout:0 2023-01-25T02:48:53.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 10 monclient: shutdown 2023-01-25T02:48:53.327 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:48:53.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff80135fa0 msgr2=0x7eff8011f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff80135fa0 0x7eff8011f4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7eff78009db0 tx=0x7eff7800c3a0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 shutdown_connections 2023-01-25T02:48:53.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7eff800a35b0 0x7eff801ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.328 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7eff80135fa0 0x7eff8011f4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 --2- 172.21.15.101:0/3831384635 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7eff800a4f50 0x7eff801caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 >> 172.21.15.101:0/3831384635 conn(0x7eff800a57b0 msgr2=0x7eff8009cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:48:53.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 shutdown_connections 2023-01-25T02:48:53.329 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.326+0000 7eff8870f700 1 -- 172.21.15.101:0/3831384635 wait complete. 2023-01-25T02:48:53.339 DEBUG:teuthology.orchestra.run.smithi101:> sudo ceph --cluster ceph osd new 42002c5e-3e02-4cd5-a2c3-4cb5fc9e1560 1 2023-01-25T02:48:53.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.460+0000 7fc76875c700 10 monclient: get_monmap_and_config 2023-01-25T02:48:53.460 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.460+0000 7fc76875c700 10 monclient: build_initial_monmap 2023-01-25T02:48:53.461 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.460+0000 7fc76875c700 10 monclient: monmap: 2023-01-25T02:48:53.461 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:48:53.461 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.461 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:53.460638+0000 2023-01-25T02:48:53.461 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:53.460638+0000 2023-01-25T02:48:53.462 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:48:53.462 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.462 INFO:teuthology.orchestra.run.smithi101.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:48:53.462 INFO:teuthology.orchestra.run.smithi101.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:48:53.462 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:48:53.462 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:48:53.463 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:48:53.463 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:48:53.463 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:48:53.463 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:48:53.463 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:48:53.464 INFO:teuthology.orchestra.run.smithi101.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:48:53.464 INFO:teuthology.orchestra.run.smithi101.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:48:53.464 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.460+0000 7fc76875c700 1 Processor -- start 2023-01-25T02:48:53.472 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 1 -- start start 2023-01-25T02:48:53.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 10 monclient: init 2023-01-25T02:48:53.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:53.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 10 monclient: _add_conns ranks=[2,4,8,7,1,10,3,0,5,6,9] 2023-01-25T02:48:53.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009f930 0x7fc76009fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.473 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 10 monclient(hunting): picked mon.noname-b con 0x7fc76009f930 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-01-25T02:48:53.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009bc10 0x7fc7600a0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 10 monclient(hunting): picked mon.noname-c con 0x7fc76009bc10 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:48:53.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.461+0000 7fc76875c700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fc76009eb60 0x7fc7601326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 10 monclient(hunting): picked mon.noname-e con 0x7fc76009eb60 addr [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] 2023-01-25T02:48:53.474 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76009f4d0 con 0x7fc76009f930 2023-01-25T02:48:53.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7600a0780 con 0x7fc76009bc10 2023-01-25T02:48:53.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.475 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 1 -- --> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fc76009c570 con 0x7fc76009eb60 2023-01-25T02:48:53.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:53.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc76875c700 10 monclient(hunting): authenticate will time out at 939.085046s 2023-01-25T02:48:53.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fc76009eb60 0x7fc7601326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009bc10 0x7fc7600a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc7664f8700 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009f930 0x7fc76009fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.476 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fc76009eb60 0x7fc7601326f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3304/0 says I am v2:172.21.15.101:52034/0 (socket says 172.21.15.101:52034) 2023-01-25T02:48:53.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 1 -- 172.21.15.101:0/1674698126 learned_addr learned my addr 172.21.15.101:0/1674698126 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:48:53.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 10 monclient(hunting): get_auth_request con 0x7fc76009eb60 auth_method 0 2023-01-25T02:48:53.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.477 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient(hunting): get_auth_request con 0x7fc76009bc10 auth_method 0 2023-01-25T02:48:53.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.478 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc7664f8700 10 monclient(hunting): get_auth_request con 0x7fc76009f930 auth_method 0 2023-01-25T02:48:53.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc7664f8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc7664f8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc7664f8700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.479 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc766cf9700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc7664f8700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.480 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc7664f8700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc7664f8700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient(hunting): handle_auth_done global_id 4126 payload 274 2023-01-25T02:48:53.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 1 -- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fc76009eb60 msgr2=0x7fc7601326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:48:53.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 1 --2- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fc76009eb60 0x7fc7601326f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.481 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 1 -- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009f930 msgr2=0x7fc76009fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:48:53.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 1 --2- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009f930 0x7fc76009fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient: _finish_hunting 0 2023-01-25T02:48:53.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 1 monclient: found mon.noname-c 2023-01-25T02:48:53.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:53.482 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3302/0 2023-01-25T02:48:53.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 1 -- 172.21.15.101:0/1674698126 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc760061b00 con 0x7fc76009bc10 2023-01-25T02:48:53.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.462+0000 7fc765cf7700 1 --2- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009bc10 0x7fc7600a0240 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc7500098a0 tx=0x7fc750009bb0 comp rx=0 tx=0).ready entity=mon.4 client_cookie=545bd82a622b98ea server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:48:53.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 1 -- 172.21.15.101:0/1674698126 <== mon.4 v2:172.21.15.19:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc750011aa0 con 0x7fc76009bc10 2023-01-25T02:48:53.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-01-25T02:48:53.483 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: dump: 2023-01-25T02:48:53.484 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.484 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.484 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.484 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.484 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.485 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.485 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.485 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.485 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.485 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.486 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.486 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.486 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.486 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.486 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.487 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: _finish_auth 0 2023-01-25T02:48:53.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: _check_auth_tickets 2023-01-25T02:48:53.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:53.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 1 -- 172.21.15.101:0/1674698126 <== mon.4 v2:172.21.15.19:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc750011c00 con 0x7fc76009bc10 2023-01-25T02:48:53.487 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 5 monclient: authenticate success, global_id 4126 2023-01-25T02:48:53.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 20 monclient: get_monmap_and_config waiting for monmap|config 2023-01-25T02:48:53.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:48:53.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 10 monclient: get_monmap_and_config success 2023-01-25T02:48:53.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 1 -- 172.21.15.101:0/1674698126 <== mon.4 v2:172.21.15.19:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc750011d70 con 0x7fc76009bc10 2023-01-25T02:48:53.488 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 4 set_mon_vals no callback set 2023-01-25T02:48:53.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-01-25T02:48:53.489 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc7654f6700 10 monclient: dump: 2023-01-25T02:48:53.489 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.489 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.490 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.490 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.490 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.490 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.490 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.491 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.491 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.491 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.491 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.491 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.492 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.492 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.492 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.492 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 10 monclient: shutdown 2023-01-25T02:48:53.492 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:48:53.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 -- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009bc10 msgr2=0x7fc7600a0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 --2- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009bc10 0x7fc7600a0240 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc7500098a0 tx=0x7fc750009bb0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 -- 172.21.15.101:0/1674698126 shutdown_connections 2023-01-25T02:48:53.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 --2- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fc76009eb60 0x7fc7601326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.493 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 --2- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009f930 0x7fc76009fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 --2- 172.21.15.101:0/1674698126 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009bc10 0x7fc7600a0240 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 -- 172.21.15.101:0/1674698126 >> 172.21.15.101:0/1674698126 conn(0x7fc7600a1e50 msgr2=0x7fc7600a2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:48:53.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 -- 172.21.15.101:0/1674698126 shutdown_connections 2023-01-25T02:48:53.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.463+0000 7fc76875c700 1 -- 172.21.15.101:0/1674698126 wait complete. 2023-01-25T02:48:53.494 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.464+0000 7fc76875c700 10 monclient: build_initial_monmap 2023-01-25T02:48:53.495 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.464+0000 7fc76875c700 10 monclient: monmap: 2023-01-25T02:48:53.495 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:48:53.495 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:48:53.495 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 0.000000 2023-01-25T02:48:53.495 INFO:teuthology.orchestra.run.smithi101.stderr:created 0.000000 2023-01-25T02:48:53.495 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:48:53.496 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 1 2023-01-25T02:48:53.496 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:48:53.496 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:48:53.496 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:48:53.497 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:48:53.497 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:48:53.497 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:48:53.497 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:48:53.497 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:48:53.497 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:48:53.498 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.464+0000 7fc76875c700 1 Processor -- start 2023-01-25T02:48:53.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.464+0000 7fc76875c700 1 -- start start 2023-01-25T02:48:53.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.464+0000 7fc76875c700 10 monclient: init 2023-01-25T02:48:53.498 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:53.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient: _add_conns ranks=[2,1,4,6,7,3,0,8,5] 2023-01-25T02:48:53.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009bc10 0x7fc7601380e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient(hunting): picked mon.noname-b con 0x7fc76009bc10 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-01-25T02:48:53.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc76009eb60 0x7fc760138620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.499 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient(hunting): picked mon.noname-f con 0x7fc76009eb60 addr [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] 2023-01-25T02:48:53.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009f930 0x7fc760138b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient(hunting): picked mon.noname-c con 0x7fc76009f930 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:48:53.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76011b680 con 0x7fc76009eb60 2023-01-25T02:48:53.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.500 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76011c000 con 0x7fc76009bc10 2023-01-25T02:48:53.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc76011bb50 con 0x7fc76009f930 2023-01-25T02:48:53.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:53.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 10 monclient(hunting): authenticate will time out at 939.088246s 2023-01-25T02:48:53.501 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc76009eb60 0x7fc760138620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc76009eb60 0x7fc760138620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:48558/0 (socket says 172.21.15.101:48558) 2023-01-25T02:48:53.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 -- 172.21.15.101:0/3311455047 learned_addr learned my addr 172.21.15.101:0/3311455047 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:48:53.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient(hunting): get_auth_request con 0x7fc76009eb60 auth_method 0 2023-01-25T02:48:53.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.502 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc766cf9700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009f930 0x7fc760138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.503 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc7664f8700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009bc10 0x7fc7601380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc766cf9700 10 monclient(hunting): get_auth_request con 0x7fc76009f930 auth_method 0 2023-01-25T02:48:53.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc766cf9700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc766cf9700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc766cf9700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.504 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc7664f8700 10 monclient(hunting): get_auth_request con 0x7fc76009bc10 auth_method 0 2023-01-25T02:48:53.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc7664f8700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc7664f8700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc7664f8700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient(hunting): handle_auth_done global_id 4132 payload 824 2023-01-25T02:48:53.505 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 -- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009f930 msgr2=0x7fc760138b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009f930 0x7fc760138b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 -- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009bc10 msgr2=0x7fc7601380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009bc10 0x7fc7601380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient: _finish_hunting 0 2023-01-25T02:48:53.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 monclient: found mon.noname-f 2023-01-25T02:48:53.506 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:53.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 10 monclient: _send_mon_message to mon.noname-f at v2:172.21.15.101:3300/0 2023-01-25T02:48:53.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 -- 172.21.15.101:0/3311455047 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7500094a0 con 0x7fc76009eb60 2023-01-25T02:48:53.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc766cf9700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009f930 0x7fc760138b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-01-25T02:48:53.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc7664f8700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009bc10 0x7fc7601380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-01-25T02:48:53.507 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc765cf7700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc76009eb60 0x7fc760138620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc750000c00 tx=0x7fc750004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:48:53.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc756ffd700 1 -- 172.21.15.101:0/3311455047 <== mon.1 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc7500043a0 con 0x7fc76009eb60 2023-01-25T02:48:53.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc756ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc756ffd700 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2023-01-25T02:48:53.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc756ffd700 10 monclient: dump: 2023-01-25T02:48:53.508 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.509 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.509 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.509 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.509 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.509 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.510 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.510 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.510 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.510 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.510 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.510 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.511 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.511 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.511 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.511 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.511 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc756ffd700 10 monclient: _finish_auth 0 2023-01-25T02:48:53.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc756ffd700 10 monclient: _check_auth_tickets 2023-01-25T02:48:53.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc756ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:53.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc756ffd700 1 -- 172.21.15.101:0/3311455047 <== mon.1 v2:172.21.15.101:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75001f070 con 0x7fc76009eb60 2023-01-25T02:48:53.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.465+0000 7fc76875c700 5 monclient: authenticate success, global_id 4132 2023-01-25T02:48:53.512 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 10 monclient: _renew_subs 2023-01-25T02:48:53.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.101:3300/0 2023-01-25T02:48:53.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76009d840 con 0x7fc76009eb60 2023-01-25T02:48:53.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc756ffd700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:48:53.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc756ffd700 1 -- 172.21.15.101:0/3311455047 <== mon.1 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc750004540 con 0x7fc76009eb60 2023-01-25T02:48:53.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc756ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.513 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc756ffd700 10 monclient: got monmap 1 from mon.f (according to old e1) 2023-01-25T02:48:53.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc764cf5700 4 set_mon_vals no callback set 2023-01-25T02:48:53.514 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc756ffd700 10 monclient: dump: 2023-01-25T02:48:53.514 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.514 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.514 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.515 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.515 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.515 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.515 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.515 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.515 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.516 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.516 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.516 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.516 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.516 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.517 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.517 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 10 monclient: _renew_subs 2023-01-25T02:48:53.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.101:3300/0 2023-01-25T02:48:53.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76009db00 con 0x7fc76009eb60 2023-01-25T02:48:53.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc756ffd700 1 -- 172.21.15.101:0/3311455047 <== mon.1 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc75002a020 con 0x7fc76009eb60 2023-01-25T02:48:53.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc756ffd700 1 -- 172.21.15.101:0/3311455047 <== mon.1 v2:172.21.15.101:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fc75002d440 con 0x7fc76009eb60 2023-01-25T02:48:53.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.101:3300/0 2023-01-25T02:48:53.518 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.466+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76009e3a0 con 0x7fc76009eb60 2023-01-25T02:48:53.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.469+0000 7fc756ffd700 1 -- 172.21.15.101:0/3311455047 <== mon.1 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7fc750015020 con 0x7fc76009eb60 2023-01-25T02:48:53.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.469+0000 7fc756ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.469+0000 7fc756ffd700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:48:53.551 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.551+0000 7fc76875c700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "42002c5e-3e02-4cd5-a2c3-4cb5fc9e1560", "id": 1}] 2023-01-25T02:48:53.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.551+0000 7fc76875c700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "42002c5e-3e02-4cd5-a2c3-4cb5fc9e1560", "id": 1}] 2023-01-25T02:48:53.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.551+0000 7fc76875c700 10 monclient: _send_mon_message to mon.f at v2:172.21.15.101:3300/0 2023-01-25T02:48:53.552 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.551+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd new", "uuid": "42002c5e-3e02-4cd5-a2c3-4cb5fc9e1560", "id": 1} v 0) v1 -- 0x7fc76006fd20 con 0x7fc76009eb60 2023-01-25T02:48:53.556 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.556+0000 7fc756ffd700 1 -- 172.21.15.101:0/3311455047 <== mon.1 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "42002c5e-3e02-4cd5-a2c3-4cb5fc9e1560", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc7500038d0 con 0x7fc76009eb60 2023-01-25T02:48:53.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.556+0000 7fc756ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "42002c5e-3e02-4cd5-a2c3-4cb5fc9e1560", "id": 1}] 2023-01-25T02:48:53.557 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.556+0000 7fc756ffd700 10 monclient: _finish_command 2 = system:0 2023-01-25T02:48:53.557 INFO:teuthology.orchestra.run.smithi101.stdout:1 2023-01-25T02:48:53.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 10 monclient: shutdown 2023-01-25T02:48:53.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:48:53.558 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc76009eb60 msgr2=0x7fc760138620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc76009eb60 0x7fc760138620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc750000c00 tx=0x7fc750004750 comp rx=0 tx=0).stop 2023-01-25T02:48:53.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 shutdown_connections 2023-01-25T02:48:53.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7fc76009f930 0x7fc760138b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7fc76009bc10 0x7fc7601380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.559 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 1 --2- 172.21.15.101:0/3311455047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc76009eb60 0x7fc760138620 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.557+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 >> 172.21.15.101:0/3311455047 conn(0x7fc7600a1e50 msgr2=0x7fc7600a3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:48:53.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.558+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 shutdown_connections 2023-01-25T02:48:53.560 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.558+0000 7fc76875c700 1 -- 172.21.15.101:0/3311455047 wait complete. 2023-01-25T02:48:53.571 DEBUG:teuthology.orchestra.run.smithi101:> sudo ceph --cluster ceph osd new 2235b6b3-ef48-49f8-925e-cbe7a454925c 2 2023-01-25T02:48:53.692 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.692+0000 7f20d88cb700 10 monclient: get_monmap_and_config 2023-01-25T02:48:53.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.692+0000 7f20d88cb700 10 monclient: build_initial_monmap 2023-01-25T02:48:53.693 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.692+0000 7f20d88cb700 10 monclient: monmap: 2023-01-25T02:48:53.693 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:48:53.693 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.693 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:53.692449+0000 2023-01-25T02:48:53.694 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:53.692449+0000 2023-01-25T02:48:53.694 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:48:53.694 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.694 INFO:teuthology.orchestra.run.smithi101.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:48:53.694 INFO:teuthology.orchestra.run.smithi101.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:48:53.695 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:48:53.695 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:48:53.695 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:48:53.695 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:48:53.695 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:48:53.696 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:48:53.696 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:48:53.696 INFO:teuthology.orchestra.run.smithi101.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:48:53.696 INFO:teuthology.orchestra.run.smithi101.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:48:53.696 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.692+0000 7f20d88cb700 1 Processor -- start 2023-01-25T02:48:53.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.692+0000 7f20d88cb700 1 -- start start 2023-01-25T02:48:53.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.692+0000 7f20d88cb700 10 monclient: init 2023-01-25T02:48:53.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:53.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient: _add_conns ranks=[9,4,10,1,2,0,3,6,8,5,7] 2023-01-25T02:48:53.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 0x7f20d409f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.706 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient(hunting): picked mon.noname-a con 0x7f20d409ed20 addr v2:172.21.15.19:3300/0 2023-01-25T02:48:53.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f20d409e3a0 0x7f20d409fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient(hunting): picked mon.noname-c con 0x7f20d409e3a0 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:48:53.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 1 --2- >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 0x7f20d40a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient(hunting): picked mon.noname-f con 0x7f20d409f0f0 addr v2:172.21.15.101:3300/0 2023-01-25T02:48:53.707 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d409d9c0 con 0x7f20d409ed20 2023-01-25T02:48:53.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 1 -- --> v2:172.21.15.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d409d400 con 0x7f20d409f0f0 2023-01-25T02:48:53.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.708 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20d409d6e0 con 0x7f20d409e3a0 2023-01-25T02:48:53.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:53.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d88cb700 10 monclient(hunting): authenticate will time out at 939.316505s 2023-01-25T02:48:53.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 1 --2- >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 0x7f20d40a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 1 --2- >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 0x7f20d40a02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:48572/0 (socket says 172.21.15.101:48572) 2023-01-25T02:48:53.709 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 1 -- 172.21.15.101:0/2484727712 learned_addr learned my addr 172.21.15.101:0/2484727712 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:48:53.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 10 monclient(hunting): get_auth_request con 0x7f20d409f0f0 auth_method 0 2023-01-25T02:48:53.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.710 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d1d9b700 1 --2- 172.21.15.101:0/2484727712 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f20d409e3a0 0x7f20d409fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 0x7f20d409f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d1d9b700 10 monclient(hunting): get_auth_request con 0x7f20d409e3a0 auth_method 0 2023-01-25T02:48:53.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d1d9b700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.711 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d1d9b700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 1 -- 172.21.15.101:0/2484727712 >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 msgr2=0x7f20d40a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-01-25T02:48:53.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 1 -- 172.21.15.101:0/2484727712 >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 msgr2=0x7f20d40a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:53.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 10 monclient(hunting): get_auth_request con 0x7f20d409ed20 auth_method 0 2023-01-25T02:48:53.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 0x7f20d40a02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:53.712 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d2d9d700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 0x7f20d40a02d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T02:48:53.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.713 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 1 -- 172.21.15.101:0/2484727712 >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 msgr2=0x7f20d409f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-01-25T02:48:53.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 1 -- 172.21.15.101:0/2484727712 >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 msgr2=0x7f20d409f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-01-25T02:48:53.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 0x7f20d409f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-01-25T02:48:53.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.714 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.693+0000 7f20d259c700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 0x7f20d409f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T02:48:53.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 10 monclient(hunting): handle_auth_done global_id 4144 payload 274 2023-01-25T02:48:53.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 1 -- 172.21.15.101:0/2484727712 >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 msgr2=0x7f20d40a02d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-01-25T02:48:53.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 0x7f20d40a02d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 1 -- 172.21.15.101:0/2484727712 >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 msgr2=0x7f20d409f850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-01-25T02:48:53.715 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 0x7f20d409f850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 10 monclient: _finish_hunting 0 2023-01-25T02:48:53.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 1 monclient: found mon.noname-c 2023-01-25T02:48:53.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:53.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3302/0 2023-01-25T02:48:53.716 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 1 -- 172.21.15.101:0/2484727712 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4061b00 con 0x7f20d409e3a0 2023-01-25T02:48:53.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d1d9b700 1 --2- 172.21.15.101:0/2484727712 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f20d409e3a0 0x7f20d409fd90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20d40a0ef0 tx=0x7f20c8009b20 comp rx=0 tx=0).ready entity=mon.4 client_cookie=8d44e65a97cd421c server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:48:53.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 1 -- 172.21.15.101:0/2484727712 <== mon.4 v2:172.21.15.19:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20c8011aa0 con 0x7f20d409e3a0 2023-01-25T02:48:53.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-01-25T02:48:53.717 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: dump: 2023-01-25T02:48:53.718 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.718 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.718 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.718 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.718 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.719 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.719 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.719 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.719 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.719 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.720 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.720 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.720 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.720 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.720 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.721 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: _finish_auth 0 2023-01-25T02:48:53.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: _check_auth_tickets 2023-01-25T02:48:53.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:53.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 1 -- 172.21.15.101:0/2484727712 <== mon.4 v2:172.21.15.19:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c8011c00 con 0x7f20d409e3a0 2023-01-25T02:48:53.721 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d88cb700 5 monclient: authenticate success, global_id 4144 2023-01-25T02:48:53.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d88cb700 20 monclient: get_monmap_and_config waiting for monmap|config 2023-01-25T02:48:53.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:48:53.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d88cb700 10 monclient: get_monmap_and_config success 2023-01-25T02:48:53.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 1 -- 172.21.15.101:0/2484727712 <== mon.4 v2:172.21.15.19:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20c8011d70 con 0x7f20d409e3a0 2023-01-25T02:48:53.722 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d88cb700 4 set_mon_vals no callback set 2023-01-25T02:48:53.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-01-25T02:48:53.723 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d159a700 10 monclient: dump: 2023-01-25T02:48:53.723 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.723 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.724 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.724 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.724 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.724 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.724 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.725 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.725 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.725 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.725 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.725 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.726 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.726 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.726 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.726 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.726 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d88cb700 10 monclient: shutdown 2023-01-25T02:48:53.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d88cb700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:48:53.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d88cb700 1 -- 172.21.15.101:0/2484727712 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f20d409e3a0 msgr2=0x7f20d409fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.694+0000 7f20d88cb700 1 --2- 172.21.15.101:0/2484727712 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f20d409e3a0 0x7f20d409fd90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20d40a0ef0 tx=0x7f20c8009b20 comp rx=0 tx=0).stop 2023-01-25T02:48:53.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 1 -- 172.21.15.101:0/2484727712 shutdown_connections 2023-01-25T02:48:53.727 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.101:3300/0 conn(0x7f20d409f0f0 0x7f20d40a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 1 --2- 172.21.15.101:0/2484727712 >> v2:172.21.15.19:3300/0 conn(0x7f20d409ed20 0x7f20d409f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 1 --2- 172.21.15.101:0/2484727712 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f20d409e3a0 0x7f20d409fd90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 1 -- 172.21.15.101:0/2484727712 >> 172.21.15.101:0/2484727712 conn(0x7f20d40a1d70 msgr2=0x7f20d40a2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:48:53.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 1 -- 172.21.15.101:0/2484727712 shutdown_connections 2023-01-25T02:48:53.728 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 1 -- 172.21.15.101:0/2484727712 wait complete. 2023-01-25T02:48:53.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 10 monclient: build_initial_monmap 2023-01-25T02:48:53.729 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.695+0000 7f20d88cb700 10 monclient: monmap: 2023-01-25T02:48:53.729 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:48:53.729 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:48:53.729 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 0.000000 2023-01-25T02:48:53.730 INFO:teuthology.orchestra.run.smithi101.stderr:created 0.000000 2023-01-25T02:48:53.730 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:48:53.730 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 1 2023-01-25T02:48:53.730 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:48:53.730 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:48:53.730 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:48:53.731 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:48:53.731 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:48:53.731 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:48:53.731 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:48:53.731 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:48:53.732 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:48:53.732 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 1 Processor -- start 2023-01-25T02:48:53.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 1 -- start start 2023-01-25T02:48:53.732 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient: init 2023-01-25T02:48:53.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient: _reopen_session rank -1 2023-01-25T02:48:53.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient: _add_conns ranks=[5,3,0,7,2,4,1,8,6] 2023-01-25T02:48:53.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f20d409e3a0 0x7f20d41ba900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient(hunting): picked mon.noname-h con 0x7f20d409e3a0 addr [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] 2023-01-25T02:48:53.733 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f20d409ed20 0x7f20d41bae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient(hunting): picked mon.noname-g con 0x7f20d409ed20 addr [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] 2023-01-25T02:48:53.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d409f0f0 0x7f20d41a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:48:53.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient(hunting): picked mon.noname-a con 0x7f20d409f0f0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-01-25T02:48:53.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.734 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41a7e20 con 0x7f20d409f0f0 2023-01-25T02:48:53.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 1 -- --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41caf50 con 0x7f20d409ed20 2023-01-25T02:48:53.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient(hunting): start opening mon connection 2023-01-25T02:48:53.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 1 -- --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20d41a8090 con 0x7f20d409e3a0 2023-01-25T02:48:53.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient(hunting): _renew_subs 2023-01-25T02:48:53.735 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d88cb700 10 monclient(hunting): authenticate will time out at 939.319656s 2023-01-25T02:48:53.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f20d409e3a0 0x7f20d41ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f20d409ed20 0x7f20d41bae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f20d409e3a0 0x7f20d41ba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3302/0 says I am v2:172.21.15.101:38218/0 (socket says 172.21.15.101:38218) 2023-01-25T02:48:53.736 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f20d409ed20 0x7f20d41bae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3301/0 says I am v2:172.21.15.101:49460/0 (socket says 172.21.15.101:49460) 2023-01-25T02:48:53.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 1 -- 172.21.15.101:0/1892957630 learned_addr learned my addr 172.21.15.101:0/1892957630 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:48:53.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 10 monclient(hunting): get_auth_request con 0x7f20d409e3a0 auth_method 0 2023-01-25T02:48:53.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.737 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 10 monclient(hunting): get_auth_request con 0x7f20d409ed20 auth_method 0 2023-01-25T02:48:53.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.738 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d1d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:48:53.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:48:53.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d259c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:48:53.739 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.696+0000 7f20d2d9d700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d409f0f0 0x7f20d41a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:48:53.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d2d9d700 10 monclient(hunting): get_auth_request con 0x7f20d409f0f0 auth_method 0 2023-01-25T02:48:53.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d2d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:48:53.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d2d9d700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:48:53.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d2d9d700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:48:53.740 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 10 monclient(hunting): handle_auth_done global_id 4107 payload 824 2023-01-25T02:48:53.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 1 -- 172.21.15.101:0/1892957630 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f20d409e3a0 msgr2=0x7f20d41ba900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f20d409e3a0 0x7f20d41ba900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 1 -- 172.21.15.101:0/1892957630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d409f0f0 msgr2=0x7f20d41a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:48:53.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d409f0f0 0x7f20d41a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:48:53.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 10 monclient: _finish_hunting 0 2023-01-25T02:48:53.741 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 1 monclient: found mon.noname-g 2023-01-25T02:48:53.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:53.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.101:3301/0 2023-01-25T02:48:53.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 1 -- 172.21.15.101:0/1892957630 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80094a0 con 0x7f20d409ed20 2023-01-25T02:48:53.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d259c700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f20d409e3a0 0x7f20d41ba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:48:53.742 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d1d9b700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f20d409ed20 0x7f20d41bae40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20d406a4b0 tx=0x7f20c8004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:48:53.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 1 -- 172.21.15.101:0/1892957630 <== mon.3 v2:172.21.15.101:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20c8003b10 con 0x7f20d409ed20 2023-01-25T02:48:53.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2023-01-25T02:48:53.743 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: dump: 2023-01-25T02:48:53.743 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.744 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.744 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.744 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.745 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.745 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.745 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.745 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.745 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.745 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.746 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.746 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.746 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.746 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.746 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.747 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: _finish_auth 0 2023-01-25T02:48:53.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: _check_auth_tickets 2023-01-25T02:48:53.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:53.747 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d88cb700 5 monclient: authenticate success, global_id 4107 2023-01-25T02:48:53.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 1 -- 172.21.15.101:0/1892957630 <== mon.3 v2:172.21.15.101:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c8003c70 con 0x7f20d409ed20 2023-01-25T02:48:53.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:48:53.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 1 -- 172.21.15.101:0/1892957630 <== mon.3 v2:172.21.15.101:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f20c8003de0 con 0x7f20d409ed20 2023-01-25T02:48:53.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d88cb700 10 monclient: _renew_subs 2023-01-25T02:48:53.748 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d88cb700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.101:3301/0 2023-01-25T02:48:53.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d406a190 con 0x7f20d409ed20 2023-01-25T02:48:53.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:48:53.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d0d99700 4 set_mon_vals no callback set 2023-01-25T02:48:53.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2023-01-25T02:48:53.749 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 10 monclient: dump: 2023-01-25T02:48:53.750 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:48:53.750 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:48:53.750 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.750 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:48:53.750 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:48:53.750 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:48:53.751 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:48:53.751 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:48:53.751 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:48:53.751 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:48:53.751 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:48:53.752 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:48:53.752 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:48:53.752 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:48:53.752 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:48:53.752 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:48:53.752 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d88cb700 10 monclient: _renew_subs 2023-01-25T02:48:53.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d88cb700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.101:3301/0 2023-01-25T02:48:53.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d409d120 con 0x7f20d409ed20 2023-01-25T02:48:53.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 1 -- 172.21.15.101:0/1892957630 <== mon.3 v2:172.21.15.101:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f20c8011d70 con 0x7f20d409ed20 2023-01-25T02:48:53.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.697+0000 7f20c2ffd700 1 -- 172.21.15.101:0/1892957630 <== mon.3 v2:172.21.15.101:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f20c80036a0 con 0x7f20d409ed20 2023-01-25T02:48:53.753 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.698+0000 7f20d88cb700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.698+0000 7f20d88cb700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.698+0000 7f20d88cb700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.101:3301/0 2023-01-25T02:48:53.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.698+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d409bb60 con 0x7f20d409ed20 2023-01-25T02:48:53.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.700+0000 7f20c2ffd700 1 -- 172.21.15.101:0/1892957630 <== mon.3 v2:172.21.15.101:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f20c8044860 con 0x7f20d409ed20 2023-01-25T02:48:53.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.700+0000 7f20c2ffd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:48:53.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.700+0000 7f20c2ffd700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:48:53.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.782+0000 7f20d88cb700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "2235b6b3-ef48-49f8-925e-cbe7a454925c", "id": 2}] 2023-01-25T02:48:53.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.782+0000 7f20d88cb700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "2235b6b3-ef48-49f8-925e-cbe7a454925c", "id": 2}] 2023-01-25T02:48:53.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.782+0000 7f20d88cb700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.101:3301/0 2023-01-25T02:48:53.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:53.782+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2235b6b3-ef48-49f8-925e-cbe7a454925c", "id": 2} v 0) v1 -- 0x7f20d41a81d0 con 0x7f20d409ed20 2023-01-25T02:48:54.242 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:54.242+0000 7f1cc2a42200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T02:48:54.310 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:54.310+0000 7f1cc2a42200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T02:48:54.394 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:54.394+0000 7f1cc2a42200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T02:48:54.578 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:54.578+0000 7f1cc2a42200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T02:48:55.357 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:55.357+0000 7f1cc2a42200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T02:48:55.496 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:55.496+0000 7f1cc2a42200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T02:48:55.716 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:55.716+0000 7f1cc2a42200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T02:48:55.786 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:55.786+0000 7f1cc2a42200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T02:48:55.921 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:55.921+0000 7f1cc2a42200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T02:48:56.181 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:56.181+0000 7f1cc2a42200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T02:48:56.361 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:56.361+0000 7f1cc2a42200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T02:48:56.498 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:56.498+0000 7f1cc2a42200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T02:48:56.612 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:56.612+0000 7f1cc2a42200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T02:48:56.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:56.696+0000 7f20c1ffb700 10 monclient: tick 2023-01-25T02:48:56.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:56.696+0000 7f20c1ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:48:56.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:56.696+0000 7f20c1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:56.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:56.696+0000 7f20c1ffb700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:48:56.750 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:56.750+0000 7f1cc2a42200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T02:48:56.828 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:56.828+0000 7f1cc2a42200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T02:48:57.045 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:57.045+0000 7f1cc2a42200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T02:48:57.122 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:57.122+0000 7f1cc2a42200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T02:48:57.696 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:57.696+0000 7f20c1ffb700 10 monclient: tick 2023-01-25T02:48:57.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:57.696+0000 7f20c1ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:48:57.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:57.696+0000 7f20c1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:57.732 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:57.732+0000 7f1cc2a42200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T02:48:57.983 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:57.983+0000 7f1cc2a42200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T02:48:58.056 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:58.056+0000 7f1cc2a42200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T02:48:58.137 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:58.137+0000 7f1cc2a42200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T02:48:58.316 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:58.316+0000 7f1cc2a42200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T02:48:58.386 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:58.386+0000 7f1cc2a42200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T02:48:58.536 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:48:58.536+0000 7f1cc2a42200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T02:48:58.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:58.696+0000 7f20c1ffb700 10 monclient: tick 2023-01-25T02:48:58.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:58.696+0000 7f20c1ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:48:58.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:58.696+0000 7f20c1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:48:59.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:59.697+0000 7f20c1ffb700 10 monclient: tick 2023-01-25T02:48:59.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:59.697+0000 7f20c1ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:48:59.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:48:59.697+0000 7f20c1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:00.697+0000 7f20c1ffb700 10 monclient: tick 2023-01-25T02:49:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:00.697+0000 7f20c1ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:49:00.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:00.697+0000 7f20c1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:01.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:01.697+0000 7f20c1ffb700 10 monclient: tick 2023-01-25T02:49:01.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:01.697+0000 7f20c1ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:49:01.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:01.697+0000 7f20c1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:02.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:02.697+0000 7f20c1ffb700 10 monclient: tick 2023-01-25T02:49:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:02.697+0000 7f20c1ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:49:02.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:02.697+0000 7f20c1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:03.615 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.615+0000 7f20c2ffd700 1 -- 172.21.15.101:0/1892957630 <== mon.3 v2:172.21.15.101:3301/0 7 ==== mgrmap(e 2) v1 ==== 42539+0+0 (secure 0 0 0) 0x7f20c800f050 con 0x7f20d409ed20 2023-01-25T02:49:03.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.621+0000 7f20c2ffd700 1 -- 172.21.15.101:0/1892957630 <== mon.3 v2:172.21.15.101:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2235b6b3-ef48-49f8-925e-cbe7a454925c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f20c8029020 con 0x7f20d409ed20 2023-01-25T02:49:03.621 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.621+0000 7f20c2ffd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "2235b6b3-ef48-49f8-925e-cbe7a454925c", "id": 2}] 2023-01-25T02:49:03.622 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.621+0000 7f20c2ffd700 10 monclient: _finish_command 2 = system:0 2023-01-25T02:49:03.622 INFO:teuthology.orchestra.run.smithi101.stdout:2 2023-01-25T02:49:03.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 10 monclient: shutdown 2023-01-25T02:49:03.623 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:03.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f20d409ed20 msgr2=0x7f20d41bae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:03.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f20d409ed20 0x7f20d41bae40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20d406a4b0 tx=0x7f20c8004750 comp rx=0 tx=0).stop 2023-01-25T02:49:03.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 shutdown_connections 2023-01-25T02:49:03.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d409f0f0 0x7f20d41a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.624 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f20d409e3a0 0x7f20d41ba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 --2- 172.21.15.101:0/1892957630 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f20d409ed20 0x7f20d41bae40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 >> 172.21.15.101:0/1892957630 conn(0x7f20d40a1d70 msgr2=0x7f20d40a3500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:03.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 shutdown_connections 2023-01-25T02:49:03.625 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.623+0000 7f20d88cb700 1 -- 172.21.15.101:0/1892957630 wait complete. 2023-01-25T02:49:03.640 DEBUG:teuthology.orchestra.run.smithi101:> sudo ceph --cluster ceph osd new 1a0c3332-393c-4a3c-8955-882e0421a27e 3 2023-01-25T02:49:03.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.765+0000 7fed81369700 10 monclient: get_monmap_and_config 2023-01-25T02:49:03.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.765+0000 7fed81369700 10 monclient: build_initial_monmap 2023-01-25T02:49:03.766 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.765+0000 7fed81369700 10 monclient: monmap: 2023-01-25T02:49:03.766 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:49:03.767 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:03.767 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:49:03.766231+0000 2023-01-25T02:49:03.767 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:49:03.766231+0000 2023-01-25T02:49:03.767 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:03.768 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:03.768 INFO:teuthology.orchestra.run.smithi101.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:49:03.768 INFO:teuthology.orchestra.run.smithi101.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:49:03.768 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:03.768 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:03.769 INFO:teuthology.orchestra.run.smithi101.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:49:03.770 INFO:teuthology.orchestra.run.smithi101.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:49:03.770 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:03.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.766+0000 7fed81369700 1 Processor -- start 2023-01-25T02:49:03.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.766+0000 7fed81369700 1 -- start start 2023-01-25T02:49:03.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.766+0000 7fed81369700 10 monclient: init 2023-01-25T02:49:03.779 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:03.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient: _add_conns ranks=[3,5,9,8,4,7,6,2,1,10,0] 2023-01-25T02:49:03.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c0a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:03.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient(hunting): picked mon.noname-g con 0x7fed7c0a3350 addr [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] 2023-01-25T02:49:03.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7fed7c0a3720 0x7fed7c0a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:03.780 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient(hunting): picked mon.noname-h con 0x7fed7c0a3720 addr [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] 2023-01-25T02:49:03.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fed7c136190 0x7fed7c136600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:03.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient(hunting): picked mon.noname-a con 0x7fed7c136190 addr v2:172.21.15.19:3300/0 2023-01-25T02:49:03.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:03.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0a1390 con 0x7fed7c136190 2023-01-25T02:49:03.781 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:03.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 1 -- --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0a1600 con 0x7fed7c0a3350 2023-01-25T02:49:03.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:03.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 1 -- --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0a18e0 con 0x7fed7c0a3720 2023-01-25T02:49:03.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:03.782 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient(hunting): authenticate will time out at 949.390204s 2023-01-25T02:49:03.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7fed7c0a3720 0x7fed7c0a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:03.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c0a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:03.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c0a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3301/0 says I am v2:172.21.15.101:39374/0 (socket says 172.21.15.101:39374) 2023-01-25T02:49:03.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7fed7c0a3720 0x7fed7c0a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3302/0 says I am v2:172.21.15.101:49870/0 (socket says 172.21.15.101:49870) 2023-01-25T02:49:03.783 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 -- 172.21.15.101:0/3346048747 learned_addr learned my addr 172.21.15.101:0/3346048747 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:49:03.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient(hunting): get_auth_request con 0x7fed7c0a3350 auth_method 0 2023-01-25T02:49:03.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:03.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:03.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:03.784 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 10 monclient(hunting): get_auth_request con 0x7fed7c0a3720 auth_method 0 2023-01-25T02:49:03.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:03.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:03.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:03.785 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:03.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:03.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:03.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:03.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:03.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7b7fe700 1 --2- 172.21.15.101:0/3346048747 >> v2:172.21.15.19:3300/0 conn(0x7fed7c136190 0x7fed7c136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:03.786 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:03.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7b7fe700 10 monclient(hunting): get_auth_request con 0x7fed7c136190 auth_method 0 2023-01-25T02:49:03.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:03.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7b7fe700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:03.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7b7fe700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:03.787 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient(hunting): handle_auth_done global_id 4116 payload 274 2023-01-25T02:49:03.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 -- 172.21.15.101:0/3346048747 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7fed7c0a3720 msgr2=0x7fed7c0a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:03.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 --2- 172.21.15.101:0/3346048747 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7fed7c0a3720 0x7fed7c0a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 -- 172.21.15.101:0/3346048747 >> v2:172.21.15.19:3300/0 conn(0x7fed7c136190 msgr2=0x7fed7c136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:03.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 --2- 172.21.15.101:0/3346048747 >> v2:172.21.15.19:3300/0 conn(0x7fed7c136190 0x7fed7c136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.788 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient: _finish_hunting 0 2023-01-25T02:49:03.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 monclient: found mon.noname-g 2023-01-25T02:49:03.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:03.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.101:3301/0 2023-01-25T02:49:03.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 -- 172.21.15.101:0/3346048747 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c061b00 con 0x7fed7c0a3350 2023-01-25T02:49:03.789 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7b7fe700 1 --2- 172.21.15.101:0/3346048747 >> v2:172.21.15.19:3300/0 conn(0x7fed7c136190 0x7fed7c136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-01-25T02:49:03.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed73fff700 1 --2- 172.21.15.101:0/3346048747 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7fed7c0a3720 0x7fed7c0a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:49:03.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7affd700 1 --2- 172.21.15.101:0/3346048747 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c0a22c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0098f0 tx=0x7fed6c009c00 comp rx=0 tx=0).ready entity=mon.3 client_cookie=edb953c70d74d281 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:03.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 1 -- 172.21.15.101:0/3346048747 <== mon.3 v2:172.21.15.101:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed6c011aa0 con 0x7fed7c0a3350 2023-01-25T02:49:03.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:03.790 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2023-01-25T02:49:03.791 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: dump: 2023-01-25T02:49:03.791 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:03.791 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:03.791 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:03.791 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:03.792 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:03.792 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:03.792 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:03.792 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:03.792 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:03.793 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:03.793 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:03.793 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:03.793 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:03.793 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:03.794 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:03.794 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:03.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: _finish_auth 0 2023-01-25T02:49:03.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: _check_auth_tickets 2023-01-25T02:49:03.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:03.794 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 5 monclient: authenticate success, global_id 4116 2023-01-25T02:49:03.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 1 -- 172.21.15.101:0/3346048747 <== mon.3 v2:172.21.15.101:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c011c00 con 0x7fed7c0a3350 2023-01-25T02:49:03.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:03.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 10 monclient: get_monmap_and_config success 2023-01-25T02:49:03.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 1 -- 172.21.15.101:0/3346048747 <== mon.3 v2:172.21.15.101:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed6c011d70 con 0x7fed7c0a3350 2023-01-25T02:49:03.795 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed81369700 4 set_mon_vals no callback set 2023-01-25T02:49:03.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:03.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: got monmap 1 from mon.g (according to old e1) 2023-01-25T02:49:03.796 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.767+0000 7fed7a7fc700 10 monclient: dump: 2023-01-25T02:49:03.796 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:03.796 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:03.797 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:03.797 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:03.797 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:03.797 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:03.797 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:03.798 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:03.798 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:03.798 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:03.798 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:03.798 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:03.799 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:03.799 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:03.799 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:03.799 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:03.799 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 10 monclient: shutdown 2023-01-25T02:49:03.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:03.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 -- 172.21.15.101:0/3346048747 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 msgr2=0x7fed7c0a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:03.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 --2- 172.21.15.101:0/3346048747 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c0a22c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0098f0 tx=0x7fed6c009c00 comp rx=0 tx=0).stop 2023-01-25T02:49:03.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 -- 172.21.15.101:0/3346048747 shutdown_connections 2023-01-25T02:49:03.800 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 --2- 172.21.15.101:0/3346048747 >> v2:172.21.15.19:3300/0 conn(0x7fed7c136190 0x7fed7c136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 --2- 172.21.15.101:0/3346048747 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c0a22c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 --2- 172.21.15.101:0/3346048747 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7fed7c0a3720 0x7fed7c0a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 -- 172.21.15.101:0/3346048747 >> 172.21.15.101:0/3346048747 conn(0x7fed7c06c080 msgr2=0x7fed7c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:03.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 -- 172.21.15.101:0/3346048747 shutdown_connections 2023-01-25T02:49:03.801 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.768+0000 7fed81369700 1 -- 172.21.15.101:0/3346048747 wait complete. 2023-01-25T02:49:03.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient: build_initial_monmap 2023-01-25T02:49:03.802 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient: monmap: 2023-01-25T02:49:03.802 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:49:03.802 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:49:03.802 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 0.000000 2023-01-25T02:49:03.803 INFO:teuthology.orchestra.run.smithi101.stderr:created 0.000000 2023-01-25T02:49:03.803 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:03.803 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 1 2023-01-25T02:49:03.803 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:49:03.803 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:49:03.804 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:03.804 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:03.804 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:03.804 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:03.804 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:03.805 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:03.805 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:03.805 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:03.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 1 Processor -- start 2023-01-25T02:49:03.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 1 -- start start 2023-01-25T02:49:03.805 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient: init 2023-01-25T02:49:03.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:03.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient: _add_conns ranks=[3,8,6,5,2,0,7,1,4] 2023-01-25T02:49:03.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c1ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:03.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient(hunting): picked mon.noname-g con 0x7fed7c0a3350 addr [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] 2023-01-25T02:49:03.806 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fed7c0a3720 0x7fed7c1cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:03.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient(hunting): picked mon.noname-e con 0x7fed7c0a3720 addr [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] 2023-01-25T02:49:03.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7fed7c136190 0x7fed7c11f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:03.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient(hunting): picked mon.noname-d con 0x7fed7c136190 addr [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] 2023-01-25T02:49:03.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:03.807 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 1 -- --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c132f40 con 0x7fed7c0a3350 2023-01-25T02:49:03.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:03.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 1 -- --> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c132200 con 0x7fed7c136190 2023-01-25T02:49:03.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:03.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 1 -- --> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c121d90 con 0x7fed7c0a3720 2023-01-25T02:49:03.808 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:03.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed81369700 10 monclient(hunting): authenticate will time out at 949.392899s 2023-01-25T02:49:03.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed7affd700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c1ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:03.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.769+0000 7fed7affd700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c1ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3301/0 says I am v2:172.21.15.101:39388/0 (socket says 172.21.15.101:39388) 2023-01-25T02:49:03.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 1 -- 172.21.15.101:0/98809111 learned_addr learned my addr 172.21.15.101:0/98809111 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:49:03.809 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient(hunting): get_auth_request con 0x7fed7c0a3350 auth_method 0 2023-01-25T02:49:03.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:03.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:03.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:03.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:03.810 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:03.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed73fff700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fed7c0a3720 0x7fed7c1cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:03.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:03.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7b7fe700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7fed7c136190 0x7fed7c11f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:03.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed73fff700 10 monclient(hunting): get_auth_request con 0x7fed7c0a3720 auth_method 0 2023-01-25T02:49:03.811 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed73fff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:03.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed73fff700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:03.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed73fff700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:03.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7b7fe700 10 monclient(hunting): get_auth_request con 0x7fed7c136190 auth_method 0 2023-01-25T02:49:03.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:03.812 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7b7fe700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:03.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7b7fe700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:03.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed73fff700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:03.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed73fff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:03.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed73fff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:03.813 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:03.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:03.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:03.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient(hunting): handle_auth_done global_id 4125 payload 824 2023-01-25T02:49:03.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 1 -- 172.21.15.101:0/98809111 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fed7c0a3720 msgr2=0x7fed7c1cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:03.814 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fed7c0a3720 0x7fed7c1cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 1 -- 172.21.15.101:0/98809111 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7fed7c136190 msgr2=0x7fed7c11f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:03.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7fed7c136190 0x7fed7c11f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient: _finish_hunting 0 2023-01-25T02:49:03.815 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 1 monclient: found mon.noname-g 2023-01-25T02:49:03.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:03.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.101:3301/0 2023-01-25T02:49:03.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 1 -- 172.21.15.101:0/98809111 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed640094a0 con 0x7fed7c0a3350 2023-01-25T02:49:03.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed7affd700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c1ca6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0a1740 tx=0x7fed6c004750 comp rx=0 tx=0).ready entity=mon.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:03.816 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 1 -- 172.21.15.101:0/98809111 <== mon.3 v2:172.21.15.101:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed6c004490 con 0x7fed7c0a3350 2023-01-25T02:49:03.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:03.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2023-01-25T02:49:03.817 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: dump: 2023-01-25T02:49:03.817 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:03.817 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:03.818 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:03.818 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:03.818 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:03.818 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:03.818 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:03.819 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:03.819 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:03.819 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:03.819 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:03.819 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:03.820 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:03.820 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:03.820 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:03.820 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:03.820 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: _finish_auth 0 2023-01-25T02:49:03.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: _check_auth_tickets 2023-01-25T02:49:03.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:03.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 1 -- 172.21.15.101:0/98809111 <== mon.3 v2:172.21.15.101:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c003b10 con 0x7fed7c0a3350 2023-01-25T02:49:03.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed81369700 5 monclient: authenticate success, global_id 4125 2023-01-25T02:49:03.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed81369700 10 monclient: _renew_subs 2023-01-25T02:49:03.821 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed81369700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.101:3301/0 2023-01-25T02:49:03.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c009550 con 0x7fed7c0a3350 2023-01-25T02:49:03.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:03.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 1 -- 172.21.15.101:0/98809111 <== mon.3 v2:172.21.15.101:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7fed6c003c80 con 0x7fed7c0a3350 2023-01-25T02:49:03.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:03.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: got monmap 1 from mon.g (according to old e1) 2023-01-25T02:49:03.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed79ffb700 4 set_mon_vals no callback set 2023-01-25T02:49:03.823 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed737fe700 10 monclient: dump: 2023-01-25T02:49:03.823 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:03.823 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:03.823 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:03.824 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:03.824 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:03.824 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:03.824 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:03.824 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:03.825 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:03.825 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:03.825 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:03.825 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:03.825 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:03.826 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:03.826 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:03.826 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:03.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed81369700 10 monclient: _renew_subs 2023-01-25T02:49:03.826 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed81369700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.101:3301/0 2023-01-25T02:49:03.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.770+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c0a0c20 con 0x7fed7c0a3350 2023-01-25T02:49:03.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.771+0000 7fed737fe700 1 -- 172.21.15.101:0/98809111 <== mon.3 v2:172.21.15.101:3301/0 4 ==== mgrmap(e 2) v1 ==== 42539+0+0 (secure 0 0 0) 0x7fed6c011d70 con 0x7fed7c0a3350 2023-01-25T02:49:03.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.771+0000 7fed737fe700 1 -- 172.21.15.101:0/98809111 <== mon.3 v2:172.21.15.101:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fed6c047c00 con 0x7fed7c0a3350 2023-01-25T02:49:03.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.771+0000 7fed81369700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:03.827 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.771+0000 7fed81369700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:03.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.771+0000 7fed81369700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.101:3301/0 2023-01-25T02:49:03.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.771+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c0a0000 con 0x7fed7c0a3350 2023-01-25T02:49:03.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.773+0000 7fed737fe700 1 -- 172.21.15.101:0/98809111 <== mon.3 v2:172.21.15.101:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7fed6c015080 con 0x7fed7c0a3350 2023-01-25T02:49:03.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.773+0000 7fed737fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:03.828 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.773+0000 7fed737fe700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:49:03.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.852+0000 7fed81369700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "1a0c3332-393c-4a3c-8955-882e0421a27e", "id": 3}] 2023-01-25T02:49:03.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.852+0000 7fed81369700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "1a0c3332-393c-4a3c-8955-882e0421a27e", "id": 3}] 2023-01-25T02:49:03.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.852+0000 7fed81369700 10 monclient: _send_mon_message to mon.g at v2:172.21.15.101:3301/0 2023-01-25T02:49:03.853 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.852+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1a0c3332-393c-4a3c-8955-882e0421a27e", "id": 3} v 0) v1 -- 0x7fed7c0a4090 con 0x7fed7c0a3350 2023-01-25T02:49:03.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.857+0000 7fed737fe700 1 -- 172.21.15.101:0/98809111 <== mon.3 v2:172.21.15.101:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a0c3332-393c-4a3c-8955-882e0421a27e", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fed6c0460b0 con 0x7fed7c0a3350 2023-01-25T02:49:03.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.857+0000 7fed737fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "1a0c3332-393c-4a3c-8955-882e0421a27e", "id": 3}] 2023-01-25T02:49:03.858 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.857+0000 7fed737fe700 10 monclient: _finish_command 2 = system:0 2023-01-25T02:49:03.858 INFO:teuthology.orchestra.run.smithi101.stdout:3 2023-01-25T02:49:03.859 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 10 monclient: shutdown 2023-01-25T02:49:03.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:03.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 msgr2=0x7fed7c1ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:03.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c1ca6a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0a1740 tx=0x7fed6c004750 comp rx=0 tx=0).stop 2023-01-25T02:49:03.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 shutdown_connections 2023-01-25T02:49:03.860 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7fed7c136190 0x7fed7c11f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7fed7c0a3350 0x7fed7c1ca6a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 --2- 172.21.15.101:0/98809111 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7fed7c0a3720 0x7fed7c1cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:03.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 >> 172.21.15.101:0/98809111 conn(0x7fed7c06c080 msgr2=0x7fed7c0a7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:03.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 shutdown_connections 2023-01-25T02:49:03.861 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.858+0000 7fed81369700 1 -- 172.21.15.101:0/98809111 wait complete. 2023-01-25T02:49:03.870 DEBUG:teuthology.orchestra.run.smithi101:> sudo ceph --cluster ceph osd new 8eee64a4-4f40-485d-8e81-8978cb8d9622 4 2023-01-25T02:49:03.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.987+0000 7f775d28c700 10 monclient: get_monmap_and_config 2023-01-25T02:49:03.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.987+0000 7f775d28c700 10 monclient: build_initial_monmap 2023-01-25T02:49:03.989 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.987+0000 7f775d28c700 10 monclient: monmap: 2023-01-25T02:49:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:49:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:49:03.988994+0000 2023-01-25T02:49:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:49:03.988994+0000 2023-01-25T02:49:03.990 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:49:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:49:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:03.991 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:03.992 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:03.992 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:03.992 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:03.992 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:03.992 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:49:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:49:03.993 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 1 Processor -- start 2023-01-25T02:49:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 1 -- start start 2023-01-25T02:49:03.993 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient: init 2023-01-25T02:49:04.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:04.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient: _add_conns ranks=[0,6,9,8,7,3,1,5,10,2,4] 2023-01-25T02:49:04.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f77580a3350 addr v1:172.21.15.19:6789/0 2023-01-25T02:49:04.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f77580a3720 0x7f77580a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.001 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient(hunting): picked mon.noname-d con 0x7f77580a3720 addr [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] 2023-01-25T02:49:04.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f775813b440 0x7f775813d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient(hunting): picked mon.noname-a con 0x7f775813b440 addr v2:172.21.15.19:3300/0 2023-01-25T02:49:04.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7758061b00 con 0x7f77580a3350 2023-01-25T02:49:04.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.002 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580a29d0 con 0x7f775813b440 2023-01-25T02:49:04.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 1 -- --> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f77580a1390 con 0x7f77580a3720 2023-01-25T02:49:04.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:04.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.988+0000 7f775d28c700 10 monclient(hunting): authenticate will time out at 949.612820s 2023-01-25T02:49:04.003 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f77580a3720 0x7f77580a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f775813b440 0x7f775813d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f775813b440 0x7f775813d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.101:52818/0 (socket says 172.21.15.101:52818) 2023-01-25T02:49:04.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7756ffd700 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f77580a3350 0x7f77580a22c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.101:51352/0 (socket says 172.21.15.101:51352) 2023-01-25T02:49:04.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 1 -- 172.21.15.101:0/109727391 learned_addr learned my addr 172.21.15.101:0/109727391 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:49:04.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f77580a3720 0x7f77580a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3303/0 says I am v2:172.21.15.101:56226/0 (socket says 172.21.15.101:56226) 2023-01-25T02:49:04.004 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 10 monclient(hunting): get_auth_request con 0x7f775813b440 auth_method 0 2023-01-25T02:49:04.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient(hunting): get_auth_request con 0x7f77580a3720 auth_method 0 2023-01-25T02:49:04.005 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.006 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient(hunting): handle_auth_done global_id 4146 payload 274 2023-01-25T02:49:04.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 1 -- 172.21.15.101:0/109727391 >> v2:172.21.15.19:3300/0 conn(0x7f775813b440 msgr2=0x7f775813d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:04.007 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 1 --2- 172.21.15.101:0/109727391 >> v2:172.21.15.19:3300/0 conn(0x7f775813b440 0x7f775813d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 1 -- 172.21.15.101:0/109727391 >> v1:172.21.15.19:6789/0 conn(0x7f77580a3350 legacy=0x7f77580a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient: _finish_hunting 0 2023-01-25T02:49:04.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 1 monclient: found mon.noname-d 2023-01-25T02:49:04.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:04.008 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 10 monclient: _send_mon_message to mon.noname-d at v2:172.21.15.19:3303/0 2023-01-25T02:49:04.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 1 -- 172.21.15.101:0/109727391 --> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580a1600 con 0x7f77580a3720 2023-01-25T02:49:04.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77577fe700 1 --2- 172.21.15.101:0/109727391 >> v2:172.21.15.19:3300/0 conn(0x7f775813b440 0x7f775813d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:49:04.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f77567fc700 1 --2- 172.21.15.101:0/109727391 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f77580a3720 0x7f77580a3b50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f774c009840 tx=0x7f774c009b50 comp rx=0 tx=0).ready entity=mon.6 client_cookie=63220afb93778d21 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:04.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 1 -- 172.21.15.101:0/109727391 <== mon.6 v2:172.21.15.19:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f774c011aa0 con 0x7f77580a3720 2023-01-25T02:49:04.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.009 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2023-01-25T02:49:04.010 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: dump: 2023-01-25T02:49:04.010 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:04.010 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.010 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.010 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.011 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.011 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.011 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.011 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.011 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.012 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.012 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.012 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.012 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.012 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.013 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.013 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: _finish_auth 0 2023-01-25T02:49:04.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:49:04.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:04.013 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 1 -- 172.21.15.101:0/109727391 <== mon.6 v2:172.21.15.19:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c011c00 con 0x7f77580a3720 2023-01-25T02:49:04.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:04.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 1 -- 172.21.15.101:0/109727391 <== mon.6 v2:172.21.15.19:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f774c011d70 con 0x7f77580a3720 2023-01-25T02:49:04.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: got monmap 1 from mon.d (according to old e1) 2023-01-25T02:49:04.014 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f7755ffb700 10 monclient: dump: 2023-01-25T02:49:04.015 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:04.015 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.015 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.015 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.015 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.015 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.016 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.016 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.016 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.016 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.016 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.017 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.017 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.017 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.017 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.017 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f775d28c700 5 monclient: authenticate success, global_id 4146 2023-01-25T02:49:04.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f775d28c700 10 monclient: get_monmap_and_config success 2023-01-25T02:49:04.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.989+0000 7f775d28c700 4 set_mon_vals no callback set 2023-01-25T02:49:04.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 10 monclient: shutdown 2023-01-25T02:49:04.018 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:04.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 1 -- 172.21.15.101:0/109727391 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f77580a3720 msgr2=0x7f77580a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 1 --2- 172.21.15.101:0/109727391 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f77580a3720 0x7f77580a3b50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f774c009840 tx=0x7f774c009b50 comp rx=0 tx=0).stop 2023-01-25T02:49:04.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 1 -- 172.21.15.101:0/109727391 shutdown_connections 2023-01-25T02:49:04.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 1 --2- 172.21.15.101:0/109727391 >> v2:172.21.15.19:3300/0 conn(0x7f775813b440 0x7f775813d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.019 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 1 --2- 172.21.15.101:0/109727391 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f77580a3720 0x7f77580a3b50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 1 -- 172.21.15.101:0/109727391 >> 172.21.15.101:0/109727391 conn(0x7f775806c080 msgr2=0x7f775806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:04.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 1 -- 172.21.15.101:0/109727391 shutdown_connections 2023-01-25T02:49:04.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 1 -- 172.21.15.101:0/109727391 wait complete. 2023-01-25T02:49:04.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.990+0000 7f775d28c700 10 monclient: build_initial_monmap 2023-01-25T02:49:04.020 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.991+0000 7f775d28c700 10 monclient: monmap: 2023-01-25T02:49:04.021 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:49:04.021 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:49:04.021 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 0.000000 2023-01-25T02:49:04.021 INFO:teuthology.orchestra.run.smithi101.stderr:created 0.000000 2023-01-25T02:49:04.021 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:04.021 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 1 2023-01-25T02:49:04.022 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:49:04.022 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:49:04.022 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:04.022 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:04.022 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:04.023 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:04.023 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:04.023 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:04.023 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:04.023 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.991+0000 7f775d28c700 1 Processor -- start 2023-01-25T02:49:04.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.991+0000 7f775d28c700 1 -- start start 2023-01-25T02:49:04.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.991+0000 7f775d28c700 10 monclient: init 2023-01-25T02:49:04.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:04.024 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient: _add_conns ranks=[4,5,7,0,1,3,6,2,8] 2023-01-25T02:49:04.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f77580a3350 0x7f77581ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient(hunting): picked mon.noname-c con 0x7f77580a3350 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:49:04.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f77580a3720 0x7f77581cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient(hunting): picked mon.noname-h con 0x7f77580a3720 addr [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] 2023-01-25T02:49:04.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f775813b440 0x7f77580a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.025 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient(hunting): picked mon.noname-i con 0x7f775813b440 addr [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] 2023-01-25T02:49:04.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f775809b9e0 con 0x7f77580a3350 2023-01-25T02:49:04.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 1 -- --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77580a0fd0 con 0x7f77580a3720 2023-01-25T02:49:04.026 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 1 -- --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f775809eab0 con 0x7f775813b440 2023-01-25T02:49:04.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:04.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient(hunting): authenticate will time out at 949.616098s 2023-01-25T02:49:04.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f77580a3720 0x7f77581cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.027 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f775813b440 0x7f77580a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f77580a3720 0x7f77581cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3302/0 says I am v2:172.21.15.101:49878/0 (socket says 172.21.15.101:49878) 2023-01-25T02:49:04.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 -- 172.21.15.101:0/2354809151 learned_addr learned my addr 172.21.15.101:0/2354809151 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:49:04.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient(hunting): get_auth_request con 0x7f77580a3720 auth_method 0 2023-01-25T02:49:04.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f775813b440 0x7f77580a7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3303/0 says I am v2:172.21.15.101:55082/0 (socket says 172.21.15.101:55082) 2023-01-25T02:49:04.028 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 10 monclient(hunting): get_auth_request con 0x7f775813b440 auth_method 0 2023-01-25T02:49:04.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.029 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f7756ffd700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f77580a3350 0x7f77581ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.030 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f7756ffd700 10 monclient(hunting): get_auth_request con 0x7f77580a3350 auth_method 0 2023-01-25T02:49:04.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f7756ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f7756ffd700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.031 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f7756ffd700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f7756ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f7756ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f7756ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient(hunting): handle_auth_done global_id 4172 payload 824 2023-01-25T02:49:04.032 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 -- 172.21.15.101:0/2354809151 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f775813b440 msgr2=0x7f77580a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f775813b440 0x7f77580a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 -- 172.21.15.101:0/2354809151 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f77580a3350 msgr2=0x7f77581ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f77580a3350 0x7f77581ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient: _finish_hunting 0 2023-01-25T02:49:04.033 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 monclient: found mon.noname-h 2023-01-25T02:49:04.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:04.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 10 monclient: _send_mon_message to mon.noname-h at v2:172.21.15.101:3302/0 2023-01-25T02:49:04.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 -- 172.21.15.101:0/2354809151 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750009b30 con 0x7f77580a3720 2023-01-25T02:49:04.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77577fe700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f775813b440 0x7f77580a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:49:04.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77567fc700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f77580a3720 0x7f77581cada0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f774c009b90 tx=0x7f774c004750 comp rx=0 tx=0).ready entity=mon.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:04.034 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 1 -- 172.21.15.101:0/2354809151 <== mon.5 v2:172.21.15.101:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f774c0036a0 con 0x7f77580a3720 2023-01-25T02:49:04.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2023-01-25T02:49:04.035 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: dump: 2023-01-25T02:49:04.035 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:04.035 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.036 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.036 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.036 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.036 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.036 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.037 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.037 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.037 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.037 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.037 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.037 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.038 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.038 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.038 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: _finish_auth 0 2023-01-25T02:49:04.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: _check_auth_tickets 2023-01-25T02:49:04.038 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:04.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 5 monclient: authenticate success, global_id 4172 2023-01-25T02:49:04.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 1 -- 172.21.15.101:0/2354809151 <== mon.5 v2:172.21.15.101:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c01f070 con 0x7f77580a3720 2023-01-25T02:49:04.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient: _renew_subs 2023-01-25T02:49:04.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.101:3302/0 2023-01-25T02:49:04.039 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774c0094a0 con 0x7f77580a3720 2023-01-25T02:49:04.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:04.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 1 -- 172.21.15.101:0/2354809151 <== mon.5 v2:172.21.15.101:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f774c003cb0 con 0x7f77580a3720 2023-01-25T02:49:04.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: got monmap 1 from mon.h (according to old e1) 2023-01-25T02:49:04.040 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f77557fa700 4 set_mon_vals no callback set 2023-01-25T02:49:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f773f7fe700 10 monclient: dump: 2023-01-25T02:49:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.041 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.042 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.042 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.042 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.042 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.042 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.043 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.043 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.043 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.043 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.043 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.043 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.044 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.044 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient: _renew_subs 2023-01-25T02:49:04.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.101:3302/0 2023-01-25T02:49:04.044 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.992+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77580a0000 con 0x7f77580a3720 2023-01-25T02:49:04.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.993+0000 7f773f7fe700 1 -- 172.21.15.101:0/2354809151 <== mon.5 v2:172.21.15.101:3302/0 4 ==== mgrmap(e 2) v1 ==== 42539+0+0 (secure 0 0 0) 0x7f774c02a020 con 0x7f77580a3720 2023-01-25T02:49:04.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.993+0000 7f775d28c700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.993+0000 7f775d28c700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.993+0000 7f775d28c700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.101:3302/0 2023-01-25T02:49:04.045 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.993+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77580a18e0 con 0x7f77580a3720 2023-01-25T02:49:04.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.993+0000 7f773f7fe700 1 -- 172.21.15.101:0/2354809151 <== mon.5 v2:172.21.15.101:3302/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f774c047a80 con 0x7f77580a3720 2023-01-25T02:49:04.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.995+0000 7f773f7fe700 1 -- 172.21.15.101:0/2354809151 <== mon.5 v2:172.21.15.101:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f774c015070 con 0x7f77580a3720 2023-01-25T02:49:04.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.995+0000 7f773f7fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.046 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:03.995+0000 7f773f7fe700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:49:04.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.076+0000 7f775d28c700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8eee64a4-4f40-485d-8e81-8978cb8d9622", "id": 4}] 2023-01-25T02:49:04.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.076+0000 7f775d28c700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8eee64a4-4f40-485d-8e81-8978cb8d9622", "id": 4}] 2023-01-25T02:49:04.077 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.076+0000 7f775d28c700 10 monclient: _send_mon_message to mon.h at v2:172.21.15.101:3302/0 2023-01-25T02:49:04.078 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.076+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_command({"prefix": "osd new", "uuid": "8eee64a4-4f40-485d-8e81-8978cb8d9622", "id": 4} v 0) v1 -- 0x7f77580a0c20 con 0x7f77580a3720 2023-01-25T02:49:04.085 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.084+0000 7f773f7fe700 1 -- 172.21.15.101:0/2354809151 <== mon.5 v2:172.21.15.101:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8eee64a4-4f40-485d-8e81-8978cb8d9622", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f774c004260 con 0x7f77580a3720 2023-01-25T02:49:04.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.085+0000 7f773f7fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8eee64a4-4f40-485d-8e81-8978cb8d9622", "id": 4}] 2023-01-25T02:49:04.086 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.085+0000 7f773f7fe700 10 monclient: _finish_command 2 = system:0 2023-01-25T02:49:04.086 INFO:teuthology.orchestra.run.smithi101.stdout:4 2023-01-25T02:49:04.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.085+0000 7f775d28c700 10 monclient: shutdown 2023-01-25T02:49:04.087 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.085+0000 7f775d28c700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:04.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.085+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f77580a3720 msgr2=0x7f77581cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.085+0000 7f775d28c700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f77580a3720 0x7f77581cada0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f774c009b90 tx=0x7f774c004750 comp rx=0 tx=0).stop 2023-01-25T02:49:04.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.086+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 shutdown_connections 2023-01-25T02:49:04.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.086+0000 7f775d28c700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f775813b440 0x7f77580a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.088 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.086+0000 7f775d28c700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f77580a3350 0x7f77581ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.086+0000 7f775d28c700 1 --2- 172.21.15.101:0/2354809151 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f77580a3720 0x7f77581cada0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.086+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 >> 172.21.15.101:0/2354809151 conn(0x7f775806c080 msgr2=0x7f7758138b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:04.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.086+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 shutdown_connections 2023-01-25T02:49:04.089 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.086+0000 7f775d28c700 1 -- 172.21.15.101:0/2354809151 wait complete. 2023-01-25T02:49:04.098 DEBUG:teuthology.orchestra.run.smithi101:> sudo ceph --cluster ceph osd new c9a38bbd-d6c9-4369-99c5-9f484a2ba38b 5 2023-01-25T02:49:04.216 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.215+0000 7f57c9889700 10 monclient: get_monmap_and_config 2023-01-25T02:49:04.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.215+0000 7f57c9889700 10 monclient: build_initial_monmap 2023-01-25T02:49:04.217 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.215+0000 7f57c9889700 10 monclient: monmap: 2023-01-25T02:49:04.217 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:49:04.217 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.217 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:49:04.216564+0000 2023-01-25T02:49:04.218 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:49:04.216564+0000 2023-01-25T02:49:04.218 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:04.218 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.218 INFO:teuthology.orchestra.run.smithi101.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:49:04.218 INFO:teuthology.orchestra.run.smithi101.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:49:04.218 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:04.219 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:04.219 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:04.219 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:04.219 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:04.219 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:04.220 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:04.220 INFO:teuthology.orchestra.run.smithi101.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:49:04.220 INFO:teuthology.orchestra.run.smithi101.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:49:04.220 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.215+0000 7f57c9889700 1 Processor -- start 2023-01-25T02:49:04.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.215+0000 7f57c9889700 1 -- start start 2023-01-25T02:49:04.221 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.215+0000 7f57c9889700 10 monclient: init 2023-01-25T02:49:04.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:04.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient: _add_conns ranks=[2,7,1,10,8,6,9,5,0,4,3] 2023-01-25T02:49:04.230 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 1 --2- >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f57c40a3350 0x7f57c40a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient(hunting): picked mon.noname-b con 0x7f57c40a3350 addr [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] 2023-01-25T02:49:04.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c40a3720 0x7f57c40a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient(hunting): picked mon.noname-i con 0x7f57c40a3720 addr [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] 2023-01-25T02:49:04.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient(hunting): picked mon.noname-f-legacy con 0x7f57c4136190 addr v1:172.21.15.101:6789/0 2023-01-25T02:49:04.231 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 1 -- --> v1:172.21.15.101:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c4061b00 con 0x7f57c4136190 2023-01-25T02:49:04.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 1 -- --> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40a1390 con 0x7f57c40a3350 2023-01-25T02:49:04.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.232 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 1 -- --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40a1600 con 0x7f57c40a3720 2023-01-25T02:49:04.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:04.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c9889700 10 monclient(hunting): authenticate will time out at 949.840333s 2023-01-25T02:49:04.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c40a3720 0x7f57c40a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c37fe700 1 --1- >> v1:172.21.15.101:6789/0 conn(0x7f57c4136190 0x7f57c4136600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.101:6789/0 says I am v1:172.21.15.101:42470/0 (socket says 172.21.15.101:42470) 2023-01-25T02:49:04.233 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c40a3720 0x7f57c40a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3303/0 says I am v2:172.21.15.101:55090/0 (socket says 172.21.15.101:55090) 2023-01-25T02:49:04.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c37fe700 1 -- 172.21.15.101:0/2066931661 learned_addr learned my addr 172.21.15.101:0/2066931661 (peer_addr_for_me v1:172.21.15.101:0/0) 2023-01-25T02:49:04.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient(hunting): get_auth_request con 0x7f57c40a3720 auth_method 0 2023-01-25T02:49:04.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.234 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c2ffd700 1 --2- 172.21.15.101:0/2066931661 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f57c40a3350 0x7f57c40a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c2ffd700 10 monclient(hunting): get_auth_request con 0x7f57c40a3350 auth_method 0 2023-01-25T02:49:04.235 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c2ffd700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c2ffd700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c1ffb700 1 -- 172.21.15.101:0/2066931661 <== mon.1 v1:172.21.15.101:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959404229 0 0) 0x7f57c4061b00 con 0x7f57c4136190 2023-01-25T02:49:04.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient(hunting): handle_auth_done global_id 4147 payload 274 2023-01-25T02:49:04.236 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 1 -- 172.21.15.101:0/2066931661 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f57c40a3350 msgr2=0x7f57c40a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:04.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 1 --2- 172.21.15.101:0/2066931661 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f57c40a3350 0x7f57c40a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 1 -- 172.21.15.101:0/2066931661 >> v1:172.21.15.101:6789/0 conn(0x7f57c4136190 legacy=0x7f57c4136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient: _finish_hunting 0 2023-01-25T02:49:04.237 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 1 monclient: found mon.noname-i 2023-01-25T02:49:04.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:04.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.101:3303/0 2023-01-25T02:49:04.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 1 -- 172.21.15.101:0/2066931661 --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40a18e0 con 0x7f57c40a3720 2023-01-25T02:49:04.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c1ffb700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1 2023-01-25T02:49:04.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.216+0000 7f57c27fc700 1 --2- 172.21.15.101:0/2066931661 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c40a3720 0x7f57c40a3b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009870 tx=0x7f57b8009b80 comp rx=0 tx=0).ready entity=mon.7 client_cookie=9b7463c6feb61786 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:04.238 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 1 -- 172.21.15.101:0/2066931661 <== mon.7 v2:172.21.15.101:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57b8011aa0 con 0x7f57c40a3720 2023-01-25T02:49:04.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2023-01-25T02:49:04.239 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: dump: 2023-01-25T02:49:04.239 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:04.239 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.240 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.240 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.240 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.240 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.240 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.241 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.241 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.241 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.241 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.241 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.242 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.242 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.242 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.242 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.242 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: _finish_auth 0 2023-01-25T02:49:04.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: _check_auth_tickets 2023-01-25T02:49:04.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:04.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 1 -- 172.21.15.101:0/2066931661 <== mon.7 v2:172.21.15.101:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8011c00 con 0x7f57c40a3720 2023-01-25T02:49:04.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 5 monclient: authenticate success, global_id 4147 2023-01-25T02:49:04.243 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 20 monclient: get_monmap_and_config waiting for monmap|config 2023-01-25T02:49:04.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:04.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 10 monclient: get_monmap_and_config success 2023-01-25T02:49:04.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 1 -- 172.21.15.101:0/2066931661 <== mon.7 v2:172.21.15.101:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57b8011d70 con 0x7f57c40a3720 2023-01-25T02:49:04.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 4 set_mon_vals no callback set 2023-01-25T02:49:04.244 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: got monmap 1 from mon.i (according to old e1) 2023-01-25T02:49:04.245 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c1ffb700 10 monclient: dump: 2023-01-25T02:49:04.245 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:04.245 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.245 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.246 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.246 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.246 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.246 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.246 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.246 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.247 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.247 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.247 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.247 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.247 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.248 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.248 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 10 monclient: shutdown 2023-01-25T02:49:04.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:04.248 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 1 -- 172.21.15.101:0/2066931661 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c40a3720 msgr2=0x7f57c40a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 1 --2- 172.21.15.101:0/2066931661 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c40a3720 0x7f57c40a3b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009870 tx=0x7f57b8009b80 comp rx=0 tx=0).stop 2023-01-25T02:49:04.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 1 -- 172.21.15.101:0/2066931661 shutdown_connections 2023-01-25T02:49:04.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 1 --2- 172.21.15.101:0/2066931661 >> [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] conn(0x7f57c40a3350 0x7f57c40a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 1 --2- 172.21.15.101:0/2066931661 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c40a3720 0x7f57c40a3b50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.249 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 1 -- 172.21.15.101:0/2066931661 >> 172.21.15.101:0/2066931661 conn(0x7f57c406c080 msgr2=0x7f57c406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:04.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 1 -- 172.21.15.101:0/2066931661 shutdown_connections 2023-01-25T02:49:04.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.217+0000 7f57c9889700 1 -- 172.21.15.101:0/2066931661 wait complete. 2023-01-25T02:49:04.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.218+0000 7f57c9889700 10 monclient: build_initial_monmap 2023-01-25T02:49:04.250 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.218+0000 7f57c9889700 10 monclient: monmap: 2023-01-25T02:49:04.250 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 0 2023-01-25T02:49:04.250 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:49:04.251 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 0.000000 2023-01-25T02:49:04.251 INFO:teuthology.orchestra.run.smithi101.stderr:created 0.000000 2023-01-25T02:49:04.251 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:04.251 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 1 2023-01-25T02:49:04.251 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:49:04.252 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:49:04.252 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:04.252 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:04.252 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:04.252 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:04.253 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.253 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.218+0000 7f57c9889700 1 Processor -- start 2023-01-25T02:49:04.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 1 -- start start 2023-01-25T02:49:04.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient: init 2023-01-25T02:49:04.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:04.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient: _add_conns ranks=[6,8,7,4,1,0,3,5,2] 2023-01-25T02:49:04.254 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f57c40a3350 0x7f57c41ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient(hunting): picked mon.noname-d con 0x7f57c40a3350 addr [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] 2023-01-25T02:49:04.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f57c40a3720 0x7f57c41cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient(hunting): picked mon.noname-e con 0x7f57c40a3720 addr [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] 2023-01-25T02:49:04.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c4136190 0x7f57c411f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient(hunting): picked mon.noname-i con 0x7f57c4136190 addr [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] 2023-01-25T02:49:04.255 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 1 -- --> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57c4123b10 con 0x7f57c40a3350 2023-01-25T02:49:04.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 1 -- --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f57c41218c0 con 0x7f57c4136190 2023-01-25T02:49:04.256 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 1 -- --> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f57c4121f60 con 0x7f57c40a3720 2023-01-25T02:49:04.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:04.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c9889700 10 monclient(hunting): authenticate will time out at 949.843710s 2023-01-25T02:49:04.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c4136190 0x7f57c411f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c4136190 0x7f57c411f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3303/0 says I am v2:172.21.15.101:55100/0 (socket says 172.21.15.101:55100) 2023-01-25T02:49:04.257 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 1 -- 172.21.15.101:0/2906322015 learned_addr learned my addr 172.21.15.101:0/2906322015 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-01-25T02:49:04.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 10 monclient(hunting): get_auth_request con 0x7f57c4136190 auth_method 0 2023-01-25T02:49:04.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.258 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c2ffd700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f57c40a3350 0x7f57c41ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c2ffd700 10 monclient(hunting): get_auth_request con 0x7f57c40a3350 auth_method 0 2023-01-25T02:49:04.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c2ffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c2ffd700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c2ffd700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.259 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c37fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c27fc700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f57c40a3720 0x7f57c41cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c27fc700 10 monclient(hunting): get_auth_request con 0x7f57c40a3720 auth_method 0 2023-01-25T02:49:04.260 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c27fc700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c27fc700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c27fc700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c2ffd700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.219+0000 7f57c2ffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.261 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c2ffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c27fc700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c27fc700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c27fc700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 10 monclient(hunting): handle_auth_done global_id 4156 payload 824 2023-01-25T02:49:04.262 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 1 -- 172.21.15.101:0/2906322015 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f57c40a3720 msgr2=0x7f57c41cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f57c40a3720 0x7f57c41cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 1 -- 172.21.15.101:0/2906322015 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f57c40a3350 msgr2=0x7f57c41ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f57c40a3350 0x7f57c41ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 10 monclient: _finish_hunting 0 2023-01-25T02:49:04.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 1 monclient: found mon.noname-i 2023-01-25T02:49:04.263 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:04.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.101:3303/0 2023-01-25T02:49:04.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 1 -- 172.21.15.101:0/2906322015 --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80094a0 con 0x7f57c4136190 2023-01-25T02:49:04.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c37fe700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c4136190 0x7f57c411f650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00cb60 tx=0x7f57bc00ce70 comp rx=0 tx=0).ready entity=mon.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:04.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 1 -- 172.21.15.101:0/2906322015 <== mon.7 v2:172.21.15.101:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57bc00ac50 con 0x7f57c4136190 2023-01-25T02:49:04.264 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2023-01-25T02:49:04.265 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: dump: 2023-01-25T02:49:04.265 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:04.265 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.265 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.266 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.266 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.266 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.266 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.266 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.267 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.267 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.267 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.267 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.267 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.268 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.268 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.268 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: _finish_auth 0 2023-01-25T02:49:04.268 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: _check_auth_tickets 2023-01-25T02:49:04.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:04.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c9889700 5 monclient: authenticate success, global_id 4156 2023-01-25T02:49:04.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 1 -- 172.21.15.101:0/2906322015 <== mon.7 v2:172.21.15.101:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00add0 con 0x7f57c4136190 2023-01-25T02:49:04.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c9889700 10 monclient: _renew_subs 2023-01-25T02:49:04.269 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c9889700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.101:3303/0 2023-01-25T02:49:04.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c40a0c20 con 0x7f57c4136190 2023-01-25T02:49:04.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:04.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 1 -- 172.21.15.101:0/2906322015 <== mon.7 v2:172.21.15.101:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57bc007540 con 0x7f57c4136190 2023-01-25T02:49:04.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: got monmap 1 from mon.i (according to old e1) 2023-01-25T02:49:04.270 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c17fa700 4 set_mon_vals no callback set 2023-01-25T02:49:04.271 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57abfff700 10 monclient: dump: 2023-01-25T02:49:04.271 INFO:teuthology.orchestra.run.smithi101.stderr:epoch 1 2023-01-25T02:49:04.271 INFO:teuthology.orchestra.run.smithi101.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.271 INFO:teuthology.orchestra.run.smithi101.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.271 INFO:teuthology.orchestra.run.smithi101.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.272 INFO:teuthology.orchestra.run.smithi101.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.272 INFO:teuthology.orchestra.run.smithi101.stderr:election_strategy: 3 2023-01-25T02:49:04.272 INFO:teuthology.orchestra.run.smithi101.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.272 INFO:teuthology.orchestra.run.smithi101.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.272 INFO:teuthology.orchestra.run.smithi101.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.273 INFO:teuthology.orchestra.run.smithi101.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.273 INFO:teuthology.orchestra.run.smithi101.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.273 INFO:teuthology.orchestra.run.smithi101.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.273 INFO:teuthology.orchestra.run.smithi101.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.273 INFO:teuthology.orchestra.run.smithi101.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.274 INFO:teuthology.orchestra.run.smithi101.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.274 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-01-25T02:49:04.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c9889700 10 monclient: _renew_subs 2023-01-25T02:49:04.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c9889700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.101:3303/0 2023-01-25T02:49:04.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.220+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c40a0000 con 0x7f57c4136190 2023-01-25T02:49:04.274 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.221+0000 7f57abfff700 1 -- 172.21.15.101:0/2906322015 <== mon.7 v2:172.21.15.101:3303/0 4 ==== mgrmap(e 2) v1 ==== 42539+0+0 (secure 0 0 0) 0x7f57bc010070 con 0x7f57c4136190 2023-01-25T02:49:04.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.221+0000 7f57c9889700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.221+0000 7f57c9889700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.221+0000 7f57c9889700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.101:3303/0 2023-01-25T02:49:04.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.221+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c4123090 con 0x7f57c4136190 2023-01-25T02:49:04.275 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.221+0000 7f57abfff700 1 -- 172.21.15.101:0/2906322015 <== mon.7 v2:172.21.15.101:3303/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f57bc045ea0 con 0x7f57c4136190 2023-01-25T02:49:04.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.223+0000 7f57abfff700 1 -- 172.21.15.101:0/2906322015 <== mon.7 v2:172.21.15.101:3303/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f57bc019270 con 0x7f57c4136190 2023-01-25T02:49:04.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.223+0000 7f57abfff700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.276 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.223+0000 7f57abfff700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:49:04.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.303+0000 7f57c9889700 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "c9a38bbd-d6c9-4369-99c5-9f484a2ba38b", "id": 5}] 2023-01-25T02:49:04.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.303+0000 7f57c9889700 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "c9a38bbd-d6c9-4369-99c5-9f484a2ba38b", "id": 5}] 2023-01-25T02:49:04.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.303+0000 7f57c9889700 10 monclient: _send_mon_message to mon.i at v2:172.21.15.101:3303/0 2023-01-25T02:49:04.305 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.303+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_command({"prefix": "osd new", "uuid": "c9a38bbd-d6c9-4369-99c5-9f484a2ba38b", "id": 5} v 0) v1 -- 0x7f57c4066210 con 0x7f57c4136190 2023-01-25T02:49:04.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.313+0000 7f57abfff700 1 -- 172.21.15.101:0/2906322015 <== mon.7 v2:172.21.15.101:3303/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c9a38bbd-d6c9-4369-99c5-9f484a2ba38b", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f57bc018020 con 0x7f57c4136190 2023-01-25T02:49:04.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.313+0000 7f57abfff700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "c9a38bbd-d6c9-4369-99c5-9f484a2ba38b", "id": 5}] 2023-01-25T02:49:04.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.313+0000 7f57abfff700 10 monclient: _finish_command 2 = system:0 2023-01-25T02:49:04.315 INFO:teuthology.orchestra.run.smithi101.stdout:5 2023-01-25T02:49:04.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 10 monclient: shutdown 2023-01-25T02:49:04.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:04.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c4136190 msgr2=0x7f57c411f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c4136190 0x7f57c411f650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00cb60 tx=0x7f57bc00ce70 comp rx=0 tx=0).stop 2023-01-25T02:49:04.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 shutdown_connections 2023-01-25T02:49:04.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f57c4136190 0x7f57c411f650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f57c40a3350 0x7f57c41ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 --2- 172.21.15.101:0/2906322015 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f57c40a3720 0x7f57c41cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.317 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 >> 172.21.15.101:0/2906322015 conn(0x7f57c406c080 msgr2=0x7f57c40a6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:04.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 shutdown_connections 2023-01-25T02:49:04.318 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-25T02:49:04.314+0000 7f57c9889700 1 -- 172.21.15.101:0/2906322015 wait complete. 2023-01-25T02:49:04.326 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T02:49:04.327 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T02:49:04.329 INFO:tasks.ceph.osd.0:Started 2023-01-25T02:49:04.330 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T02:49:04.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T02:49:04.332 INFO:tasks.ceph.osd.1:Started 2023-01-25T02:49:04.333 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T02:49:04.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T02:49:04.335 INFO:tasks.ceph.osd.2:Started 2023-01-25T02:49:04.335 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-25T02:49:04.336 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-25T02:49:04.338 INFO:tasks.ceph.osd.3:Started 2023-01-25T02:49:04.338 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-25T02:49:04.339 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-25T02:49:04.341 INFO:tasks.ceph.osd.4:Started 2023-01-25T02:49:04.341 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-25T02:49:04.342 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-25T02:49:04.344 INFO:tasks.ceph.osd.5:Started 2023-01-25T02:49:04.344 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T02:49:04.418 INFO:tasks.ceph.osd.3.smithi101.stderr:2023-01-25T02:49:04.417+0000 7f8bad8745c0 -1 Falling back to public interface 2023-01-25T02:49:04.419 INFO:tasks.ceph.osd.4.smithi101.stderr:2023-01-25T02:49:04.418+0000 7f6003ff35c0 -1 Falling back to public interface 2023-01-25T02:49:04.425 INFO:tasks.ceph.osd.5.smithi101.stderr:2023-01-25T02:49:04.424+0000 7eff3ba9e5c0 -1 Falling back to public interface 2023-01-25T02:49:04.426 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-01-25T02:49:04.425+0000 7fca258325c0 -1 Falling back to public interface 2023-01-25T02:49:04.426 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-01-25T02:49:04.425+0000 7ff3ff0595c0 -1 Falling back to public interface 2023-01-25T02:49:04.427 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-01-25T02:49:04.426+0000 7fca1cbe0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-01-25T02:49:04.427 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-01-25T02:49:04.426+0000 7f7cb19345c0 -1 Falling back to public interface 2023-01-25T02:49:04.431 INFO:tasks.ceph.osd.3.smithi101.stderr:2023-01-25T02:49:04.430+0000 7f8bad8745c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:49:04.431 INFO:tasks.ceph.osd.4.smithi101.stderr:2023-01-25T02:49:04.430+0000 7f6003ff35c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:49:04.432 INFO:tasks.ceph.osd.3.smithi101.stderr:2023-01-25T02:49:04.430+0000 7f8bad8745c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.432 INFO:tasks.ceph.osd.3.smithi101.stderr:2023-01-25T02:49:04.430+0000 7f8bad8745c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.432 INFO:tasks.ceph.osd.4.smithi101.stderr:2023-01-25T02:49:04.430+0000 7f6003ff35c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.432 INFO:tasks.ceph.osd.4.smithi101.stderr:2023-01-25T02:49:04.430+0000 7f6003ff35c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.437 INFO:tasks.ceph.osd.4.smithi101.stderr:2023-01-25T02:49:04.436+0000 7f6003ff35c0 -1 osd.4 0 log_to_monitors true 2023-01-25T02:49:04.438 INFO:tasks.ceph.osd.5.smithi101.stderr:2023-01-25T02:49:04.437+0000 7eff3ba9e5c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:49:04.438 INFO:tasks.ceph.osd.5.smithi101.stderr:2023-01-25T02:49:04.437+0000 7eff3ba9e5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.439 INFO:tasks.ceph.osd.5.smithi101.stderr:2023-01-25T02:49:04.437+0000 7eff3ba9e5c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.440 INFO:tasks.ceph.osd.3.smithi101.stderr:2023-01-25T02:49:04.439+0000 7f8bad8745c0 -1 osd.3 0 log_to_monitors true 2023-01-25T02:49:04.441 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-01-25T02:49:04.441+0000 7ff3ff0595c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:49:04.441 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-01-25T02:49:04.441+0000 7ff3ff0595c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.442 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-01-25T02:49:04.441+0000 7ff3ff0595c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.442 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-01-25T02:49:04.442+0000 7fca258325c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:49:04.443 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-01-25T02:49:04.442+0000 7fca258325c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.443 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-01-25T02:49:04.442+0000 7fca258325c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.445 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-01-25T02:49:04.444+0000 7f7cb19345c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2023-01-25T02:49:04.445 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-01-25T02:49:04.444+0000 7f7cb19345c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.445 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-01-25T02:49:04.444+0000 7f7cb19345c0 -1 journal do_read_entry(8192): bad header magic 2023-01-25T02:49:04.446 INFO:tasks.ceph.osd.5.smithi101.stderr:2023-01-25T02:49:04.444+0000 7eff3ba9e5c0 -1 osd.5 0 log_to_monitors true 2023-01-25T02:49:04.451 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-01-25T02:49:04.450+0000 7ff3ff0595c0 -1 osd.1 0 log_to_monitors true 2023-01-25T02:49:04.451 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-01-25T02:49:04.451+0000 7f7cb19345c0 -1 osd.2 0 log_to_monitors true 2023-01-25T02:49:04.452 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-01-25T02:49:04.452+0000 7fca258325c0 -1 osd.0 0 log_to_monitors true 2023-01-25T02:49:04.502 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.502+0000 7f5a222fa700 10 monclient: get_monmap_and_config 2023-01-25T02:49:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.502+0000 7f5a222fa700 10 monclient: build_initial_monmap 2023-01-25T02:49:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.502+0000 7f5a222fa700 10 monclient: monmap: 2023-01-25T02:49:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-01-25T02:49:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:49:04.502840+0000 2023-01-25T02:49:04.504 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:49:04.502840+0000 2023-01-25T02:49:04.504 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:04.504 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:04.504 INFO:teuthology.orchestra.run.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:49:04.504 INFO:teuthology.orchestra.run.smithi019.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:49:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:49:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:49:04.506 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:49:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.502+0000 7f5a222fa700 1 Processor -- start 2023-01-25T02:49:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.502+0000 7f5a222fa700 1 -- start start 2023-01-25T02:49:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.502+0000 7f5a222fa700 10 monclient: init 2023-01-25T02:49:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient: _add_conns ranks=[3,10,5,7,2,8,1,6,0,9,4] 2023-01-25T02:49:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f5a1c0a3350 0x7f5a1c0a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient(hunting): picked mon.noname-g con 0x7f5a1c0a3350 addr [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] 2023-01-25T02:49:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 1 --2- >> v2:172.21.15.101:3300/0 conn(0x7f5a1c0a3720 0x7f5a1c0a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient(hunting): picked mon.noname-f con 0x7f5a1c0a3720 addr v2:172.21.15.101:3300/0 2023-01-25T02:49:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c136600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient(hunting): picked mon.noname-h con 0x7f5a1c136190 addr [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] 2023-01-25T02:49:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 1 -- --> v2:172.21.15.101:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1c0a1390 con 0x7f5a1c0a3720 2023-01-25T02:49:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 1 -- --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0a1600 con 0x7f5a1c0a3350 2023-01-25T02:49:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 1 -- --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0a18e0 con 0x7f5a1c136190 2023-01-25T02:49:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a222fa700 10 monclient(hunting): authenticate will time out at 954.028477s 2023-01-25T02:49:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a20897700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1b7fe700 1 --2- >> v2:172.21.15.101:3300/0 conn(0x7f5a1c0a3720 0x7f5a1c0a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1bfff700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f5a1c0a3350 0x7f5a1c0a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a20897700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c136600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3302/0 says I am v2:172.21.15.19:50676/0 (socket says 172.21.15.19:50676) 2023-01-25T02:49:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a20897700 1 -- 172.21.15.19:0/3929556431 learned_addr learned my addr 172.21.15.19:0/3929556431 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-01-25T02:49:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a20897700 10 monclient(hunting): get_auth_request con 0x7f5a1c136190 auth_method 0 2023-01-25T02:49:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a20897700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a20897700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a20897700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1bfff700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a3350 auth_method 0 2023-01-25T02:49:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1bfff700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1bfff700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1b7fe700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a3720 auth_method 0 2023-01-25T02:49:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.521 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1b7fe700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1b7fe700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.503+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a20897700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a20897700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a20897700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_done global_id 4161 payload 274 2023-01-25T02:49:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 1 -- 172.21.15.19:0/3929556431 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 msgr2=0x7f5a1c136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 1 --2- 172.21.15.19:0/3929556431 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 1 -- 172.21.15.19:0/3929556431 >> v2:172.21.15.101:3300/0 conn(0x7f5a1c0a3720 msgr2=0x7f5a1c0a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 1 --2- 172.21.15.19:0/3929556431 >> v2:172.21.15.101:3300/0 conn(0x7f5a1c0a3720 0x7f5a1c0a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.524 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 10 monclient: _finish_hunting 0 2023-01-25T02:49:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 1 monclient: found mon.noname-g 2023-01-25T02:49:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 10 monclient: _send_mon_message to mon.noname-g at v2:172.21.15.101:3301/0 2023-01-25T02:49:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 1 -- 172.21.15.19:0/3929556431 --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c061b00 con 0x7f5a1c0a3350 2023-01-25T02:49:04.525 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1b7fe700 1 --2- 172.21.15.19:0/3929556431 >> v2:172.21.15.101:3300/0 conn(0x7f5a1c0a3720 0x7f5a1c0a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:49:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1bfff700 1 --2- 172.21.15.19:0/3929556431 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f5a1c0a3350 0x7f5a1c0a22c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009870 tx=0x7f5a0c009b80 comp rx=0 tx=0).ready entity=mon.3 client_cookie=c4d24db08290693 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 1 -- 172.21.15.19:0/3929556431 <== mon.3 v2:172.21.15.101:3301/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a0c011aa0 con 0x7f5a1c0a3350 2023-01-25T02:49:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2023-01-25T02:49:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: dump: 2023-01-25T02:49:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:49:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:04.528 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.528 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.528 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.528 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.528 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.529 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.529 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.529 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.529 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.529 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:49:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: _finish_auth 0 2023-01-25T02:49:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: _check_auth_tickets 2023-01-25T02:49:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a222fa700 5 monclient: authenticate success, global_id 4161 2023-01-25T02:49:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 1 -- 172.21.15.19:0/3929556431 <== mon.3 v2:172.21.15.101:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c011c00 con 0x7f5a1c0a3350 2023-01-25T02:49:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a222fa700 10 monclient: get_monmap_and_config success 2023-01-25T02:49:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 1 -- 172.21.15.19:0/3929556431 <== mon.3 v2:172.21.15.101:3301/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a0c011d70 con 0x7f5a1c0a3350 2023-01-25T02:49:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a222fa700 4 set_mon_vals no callback set 2023-01-25T02:49:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: got monmap 1 from mon.g (according to old e1) 2023-01-25T02:49:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a1affd700 10 monclient: dump: 2023-01-25T02:49:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:49:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.535 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:49:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a222fa700 10 monclient: shutdown 2023-01-25T02:49:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a222fa700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a222fa700 1 -- 172.21.15.19:0/3929556431 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f5a1c0a3350 msgr2=0x7f5a1c0a22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.504+0000 7f5a222fa700 1 --2- 172.21.15.19:0/3929556431 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f5a1c0a3350 0x7f5a1c0a22c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009870 tx=0x7f5a0c009b80 comp rx=0 tx=0).stop 2023-01-25T02:49:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 1 -- 172.21.15.19:0/3929556431 shutdown_connections 2023-01-25T02:49:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 1 --2- 172.21.15.19:0/3929556431 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 1 --2- 172.21.15.19:0/3929556431 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f5a1c0a3350 0x7f5a1c0a22c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 1 --2- 172.21.15.19:0/3929556431 >> v2:172.21.15.101:3300/0 conn(0x7f5a1c0a3720 0x7f5a1c0a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 1 -- 172.21.15.19:0/3929556431 >> 172.21.15.19:0/3929556431 conn(0x7f5a1c06c080 msgr2=0x7f5a1c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 1 -- 172.21.15.19:0/3929556431 shutdown_connections 2023-01-25T02:49:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 1 -- 172.21.15.19:0/3929556431 wait complete. 2023-01-25T02:49:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 10 monclient: build_initial_monmap 2023-01-25T02:49:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.505+0000 7f5a222fa700 10 monclient: monmap: 2023-01-25T02:49:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 0 2023-01-25T02:49:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:49:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 0.000000 2023-01-25T02:49:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:created 0.000000 2023-01-25T02:49:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 1 2023-01-25T02:49:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:49:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:49:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:04.540 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:49:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 1 Processor -- start 2023-01-25T02:49:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 1 -- start start 2023-01-25T02:49:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient: init 2023-01-25T02:49:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient: _add_conns ranks=[8,4,5,7,0,2,1,6,3] 2023-01-25T02:49:04.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f5a1c0a3350 0x7f5a1c1ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient(hunting): picked mon.noname-e con 0x7f5a1c0a3350 addr [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] 2023-01-25T02:49:04.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f5a1c0a3720 0x7f5a1c1cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient(hunting): picked mon.noname-c con 0x7f5a1c0a3720 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:49:04.542 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c11f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:04.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient(hunting): picked mon.noname-h con 0x7f5a1c136190 addr [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] 2023-01-25T02:49:04.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c09b9e0 con 0x7f5a1c0a3720 2023-01-25T02:49:04.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 1 -- --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c123480 con 0x7f5a1c136190 2023-01-25T02:49:04.543 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:04.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 1 -- --> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0a0fd0 con 0x7f5a1c0a3350 2023-01-25T02:49:04.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:04.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a222fa700 10 monclient(hunting): authenticate will time out at 954.031790s 2023-01-25T02:49:04.544 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f5a1c0a3350 0x7f5a1c1ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f5a1c0a3350 0x7f5a1c1ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3304/0 says I am v2:172.21.15.19:59006/0 (socket says 172.21.15.19:59006) 2023-01-25T02:49:04.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 1 -- 172.21.15.19:0/3951547301 learned_addr learned my addr 172.21.15.19:0/3951547301 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-01-25T02:49:04.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a3350 auth_method 0 2023-01-25T02:49:04.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.545 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1b7fe700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f5a1c0a3720 0x7f5a1c1cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1b7fe700 10 monclient(hunting): get_auth_request con 0x7f5a1c0a3720 auth_method 0 2023-01-25T02:49:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1b7fe700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1b7fe700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1b7fe700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.506+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1bfff700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a20897700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c11f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a20897700 10 monclient(hunting): get_auth_request con 0x7f5a1c136190 auth_method 0 2023-01-25T02:49:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a20897700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a20897700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a20897700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 10 monclient(hunting): handle_auth_done global_id 4207 payload 824 2023-01-25T02:49:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 1 -- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f5a1c0a3350 msgr2=0x7f5a1c1ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f5a1c0a3350 0x7f5a1c1ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 1 -- 172.21.15.19:0/3951547301 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 msgr2=0x7f5a1c11f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c11f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 10 monclient: _finish_hunting 0 2023-01-25T02:49:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 1 monclient: found mon.noname-c 2023-01-25T02:49:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3302/0 2023-01-25T02:49:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 1 -- 172.21.15.19:0/3951547301 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a040094a0 con 0x7f5a1c0a3720 2023-01-25T02:49:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1bfff700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f5a1c0a3350 0x7f5a1c1ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:49:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a20897700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c11f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-01-25T02:49:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1b7fe700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f5a1c0a3720 0x7f5a1c1cabe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e980 tx=0x7f5a0400ec90 comp rx=0 tx=0).ready entity=mon.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 1 -- 172.21.15.19:0/3951547301 <== mon.4 v2:172.21.15.19:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a0400eed0 con 0x7f5a1c0a3720 2023-01-25T02:49:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-01-25T02:49:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: dump: 2023-01-25T02:49:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:49:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.553 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.553 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.553 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.553 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:04.553 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.554 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.554 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.554 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.554 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.554 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.554 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.555 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.555 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.555 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:49:04.555 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: _finish_auth 0 2023-01-25T02:49:04.555 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: _check_auth_tickets 2023-01-25T02:49:04.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:04.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 1 -- 172.21.15.19:0/3951547301 <== mon.4 v2:172.21.15.19:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0400ccc0 con 0x7f5a1c0a3720 2023-01-25T02:49:04.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a222fa700 5 monclient: authenticate success, global_id 4207 2023-01-25T02:49:04.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a222fa700 10 monclient: _renew_subs 2023-01-25T02:49:04.556 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a222fa700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3302/0 2023-01-25T02:49:04.557 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c0094a0 con 0x7f5a1c0a3720 2023-01-25T02:49:04.557 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:04.557 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 1 -- 172.21.15.19:0/3951547301 <== mon.4 v2:172.21.15.19:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a0400ce30 con 0x7f5a1c0a3720 2023-01-25T02:49:04.557 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:04.557 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-01-25T02:49:04.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a1a7fc700 4 set_mon_vals no callback set 2023-01-25T02:49:04.558 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a18ff9700 10 monclient: dump: 2023-01-25T02:49:04.558 INFO:teuthology.orchestra.run.smithi019.stderr:epoch 1 2023-01-25T02:49:04.558 INFO:teuthology.orchestra.run.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:04.558 INFO:teuthology.orchestra.run.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.559 INFO:teuthology.orchestra.run.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:04.559 INFO:teuthology.orchestra.run.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:04.559 INFO:teuthology.orchestra.run.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:04.559 INFO:teuthology.orchestra.run.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:04.559 INFO:teuthology.orchestra.run.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:04.559 INFO:teuthology.orchestra.run.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:04.560 INFO:teuthology.orchestra.run.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:04.561 INFO:teuthology.orchestra.run.smithi019.stderr: 2023-01-25T02:49:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a222fa700 10 monclient: _renew_subs 2023-01-25T02:49:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a222fa700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3302/0 2023-01-25T02:49:04.561 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.507+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c0a0c20 con 0x7f5a1c0a3720 2023-01-25T02:49:04.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.508+0000 7f5a222fa700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.508+0000 7f5a222fa700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.508+0000 7f5a222fa700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3302/0 2023-01-25T02:49:04.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.508+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c0a0000 con 0x7f5a1c0a3720 2023-01-25T02:49:04.562 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.508+0000 7f5a18ff9700 1 -- 172.21.15.19:0/3951547301 <== mon.4 v2:172.21.15.19:3302/0 4 ==== mgrmap(e 2) v1 ==== 42539+0+0 (secure 0 0 0) 0x7f5a04019030 con 0x7f5a1c0a3720 2023-01-25T02:49:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.508+0000 7f5a18ff9700 1 -- 172.21.15.19:0/3951547301 <== mon.4 v2:172.21.15.19:3302/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f5a04046820 con 0x7f5a1c0a3720 2023-01-25T02:49:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.510+0000 7f5a18ff9700 1 -- 172.21.15.19:0/3951547301 <== mon.4 v2:172.21.15.19:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f5a0401b590 con 0x7f5a1c0a3720 2023-01-25T02:49:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.510+0000 7f5a18ff9700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:04.563 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.510+0000 7f5a18ff9700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:49:04.583 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.582+0000 7f5a222fa700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-01-25T02:49:04.583 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.582+0000 7f5a222fa700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-01-25T02:49:04.583 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.582+0000 7f5a222fa700 10 monclient: _send_mon_message to mon.c at v2:172.21.15.19:3302/0 2023-01-25T02:49:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.582+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a1c0a4090 con 0x7f5a1c0a3720 2023-01-25T02:49:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a18ff9700 1 -- 172.21.15.19:0/3951547301 <== mon.4 v2:172.21.15.19:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+5582 (secure 0 0 0) 0x7f5a04017070 con 0x7f5a1c0a3720 2023-01-25T02:49:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a18ff9700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-01-25T02:49:04.584 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a18ff9700 10 monclient: _finish_command 2 = system:0 2023-01-25T02:49:04.585 INFO:teuthology.orchestra.run.smithi019.stdout: 2023-01-25T02:49:04.585 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"fsid":"079d300e-8dda-42d1-8c3c-224268975a31","created":"2023-01-25T02:48:52.592043+0000","modified":"2023-01-25T02:49:04.305389+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T02:49:04.305389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"635bff03-56e6-4b89-821e-97ea356fb68c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42002c5e-3e02-4cd5-a2c3-4cb5fc9e1560","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2235b6b3-ef48-49f8-925e-cbe7a454925c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1a0c3332-393c-4a3c-8955-882e0421a27e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8eee64a4-4f40-485d-8e81-8978cb8d9622","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c9a38bbd-d6c9-4369-99c5-9f484a2ba38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T02:49:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 10 monclient: shutdown 2023-01-25T02:49:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:04.586 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f5a1c0a3720 msgr2=0x7f5a1c1cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:04.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f5a1c0a3720 0x7f5a1c1cabe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400e980 tx=0x7f5a0400ec90 comp rx=0 tx=0).stop 2023-01-25T02:49:04.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 shutdown_connections 2023-01-25T02:49:04.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f5a1c0a3350 0x7f5a1c1ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.587 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f5a1c136190 0x7f5a1c11f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 1 --2- 172.21.15.19:0/3951547301 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f5a1c0a3720 0x7f5a1c1cabe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.583+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 >> 172.21.15.19:0/3951547301 conn(0x7f5a1c06c080 msgr2=0x7f5a1c0a7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.584+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 shutdown_connections 2023-01-25T02:49:04.588 INFO:teuthology.orchestra.run.smithi019.stderr:2023-01-25T02:49:04.584+0000 7f5a222fa700 1 -- 172.21.15.19:0/3951547301 wait complete. 2023-01-25T02:49:04.596 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T02:49:04.596 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T02:49:04.628 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:49:04.626+0000 7f1c7c5bb700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-25T02:49:04.629 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:49:04.627+0000 7f1c7c5bb700 -1 mgr.server handle_report got status from non-daemon mon.h 2023-01-25T02:49:04.629 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:49:04.627+0000 7f1c7c5bb700 -1 mgr.server handle_report got status from non-daemon mon.i 2023-01-25T02:49:04.629 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:49:04.627+0000 7f1c7c5bb700 -1 mgr.server handle_report got status from non-daemon mon.g 2023-01-25T02:49:04.629 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:49:04.628+0000 7f1c7c5bb700 -1 mgr.server handle_report got status from non-daemon mon.f 2023-01-25T02:49:04.630 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:49:04.628+0000 7f1c7c5bb700 -1 mgr.server handle_report got status from non-daemon mon.e 2023-01-25T02:49:04.630 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:49:04.629+0000 7f1c7c5bb700 -1 mgr.server handle_report got status from non-daemon mon.d 2023-01-25T02:49:04.631 INFO:tasks.ceph.mgr.x.smithi101.stderr:2023-01-25T02:49:04.629+0000 7f1c7c5bb700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-25T02:49:05.201 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T02:49:05.325 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.324+0000 7f3d51107700 10 monclient: get_monmap_and_config 2023-01-25T02:49:05.325 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.324+0000 7f3d51107700 10 monclient: build_initial_monmap 2023-01-25T02:49:05.326 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.324+0000 7f3d51107700 10 monclient: monmap: 2023-01-25T02:49:05.326 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-01-25T02:49:05.326 INFO:teuthology.misc.health.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:05.326 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-01-25T02:49:05.325542+0000 2023-01-25T02:49:05.327 INFO:teuthology.misc.health.smithi019.stderr:created 2023-01-25T02:49:05.325542+0000 2023-01-25T02:49:05.327 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:05.327 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:05.328 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:49:05.328 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:49:05.328 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:05.328 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:05.329 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:05.329 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:05.329 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:05.329 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:05.330 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:05.330 INFO:teuthology.misc.health.smithi019.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:49:05.330 INFO:teuthology.misc.health.smithi019.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:49:05.330 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:05.339 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 1 Processor -- start 2023-01-25T02:49:05.339 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 1 -- start start 2023-01-25T02:49:05.339 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient: init 2023-01-25T02:49:05.339 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:05.340 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient: _add_conns ranks=[0,5,7,10,8,1,2,6,3,4,9] 2023-01-25T02:49:05.340 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient(hunting): picked mon.noname-a-legacy con 0x7f3d4c136230 addr v1:172.21.15.19:6789/0 2023-01-25T02:49:05.340 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 1 --2- >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f3d4c136d10 0x7f3d4c0a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:05.340 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient(hunting): picked mon.noname-h con 0x7f3d4c136d10 addr [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] 2023-01-25T02:49:05.340 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 1 --2- >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3d4c13b470 0x7f3d4c13d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:05.341 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient(hunting): picked mon.noname-i con 0x7f3d4c13b470 addr [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] 2023-01-25T02:49:05.341 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c061b00 con 0x7f3d4c136230 2023-01-25T02:49:05.341 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:05.341 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 1 -- --> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c1370e0 con 0x7f3d4c136d10 2023-01-25T02:49:05.341 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:05.342 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 1 -- --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0a4ca0 con 0x7f3d4c13b470 2023-01-25T02:49:05.342 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:05.342 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.325+0000 7f3d51107700 10 monclient(hunting): authenticate will time out at 954.850949s 2023-01-25T02:49:05.342 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4ad9d700 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3d4c136230 0x7f3d4c136600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48312/0 (socket says 172.21.15.19:48312) 2023-01-25T02:49:05.342 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4ad9d700 1 -- 172.21.15.19:0/2100183427 learned_addr learned my addr 172.21.15.19:0/2100183427 (peer_addr_for_me v1:172.21.15.19:0/0) 2023-01-25T02:49:05.343 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 1 --2- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3d4c13b470 0x7f3d4c13d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:05.343 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4a59c700 1 --2- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f3d4c136d10 0x7f3d4c0a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:05.343 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient(hunting): get_auth_request con 0x7f3d4c13b470 auth_method 0 2023-01-25T02:49:05.343 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:05.343 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:05.344 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:05.344 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4a59c700 10 monclient(hunting): get_auth_request con 0x7f3d4c136d10 auth_method 0 2023-01-25T02:49:05.344 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:05.345 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4a59c700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:05.345 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4a59c700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:05.345 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 1 -- 172.21.15.19:0/2100183427 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863790878 0 0) 0x7f3d4c061b00 con 0x7f3d4c136230 2023-01-25T02:49:05.345 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:05.345 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:05.346 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:05.346 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient(hunting): _negotiate 2023-01-25T02:49:05.346 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:05.346 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:05.347 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient(hunting): my global_id is 14202 2023-01-25T02:49:05.347 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 1 -- 172.21.15.19:0/2100183427 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d380035b0 con 0x7f3d4c136230 2023-01-25T02:49:05.347 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:05.348 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:05.348 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:05.348 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient(hunting): handle_auth_done global_id 4210 payload 274 2023-01-25T02:49:05.349 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 1 -- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f3d4c136d10 msgr2=0x7f3d4c0a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:05.349 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 1 --2- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f3d4c136d10 0x7f3d4c0a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.349 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 1 -- 172.21.15.19:0/2100183427 >> v1:172.21.15.19:6789/0 conn(0x7f3d4c136230 legacy=0x7f3d4c136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:05.349 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient: _finish_hunting 0 2023-01-25T02:49:05.350 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 1 monclient: found mon.noname-i 2023-01-25T02:49:05.350 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:05.350 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 10 monclient: _send_mon_message to mon.noname-i at v2:172.21.15.101:3303/0 2023-01-25T02:49:05.350 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 1 -- 172.21.15.19:0/2100183427 --> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c09c720 con 0x7f3d4c13b470 2023-01-25T02:49:05.350 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d4b59e700 1 --2- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3d4c13b470 0x7f3d4c13d850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009dc0 tx=0x7f3d4400b330 comp rx=0 tx=0).ready entity=mon.7 client_cookie=2e99e9433adb6977 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:05.351 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 1 -- 172.21.15.19:0/2100183427 <== mon.7 v2:172.21.15.101:3303/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d4400e070 con 0x7f3d4c13b470 2023-01-25T02:49:05.351 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:05.351 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2023-01-25T02:49:05.351 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: dump: 2023-01-25T02:49:05.352 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-01-25T02:49:05.352 INFO:teuthology.misc.health.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:05.352 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:05.352 INFO:teuthology.misc.health.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:05.353 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:05.353 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:05.353 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:05.353 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:05.353 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:05.354 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:05.354 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:05.354 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:05.354 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:05.354 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:05.355 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:05.355 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:05.355 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: _finish_auth 0 2023-01-25T02:49:05.355 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: _check_auth_tickets 2023-01-25T02:49:05.355 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:05.355 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d51107700 5 monclient: authenticate success, global_id 4210 2023-01-25T02:49:05.356 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d51107700 20 monclient: get_monmap_and_config waiting for monmap|config 2023-01-25T02:49:05.356 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 1 -- 172.21.15.19:0/2100183427 <== mon.7 v2:172.21.15.101:3303/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d44012430 con 0x7f3d4c13b470 2023-01-25T02:49:05.356 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:05.356 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d51107700 10 monclient: get_monmap_and_config success 2023-01-25T02:49:05.357 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d51107700 4 set_mon_vals no callback set 2023-01-25T02:49:05.357 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 1 -- 172.21.15.19:0/2100183427 <== mon.7 v2:172.21.15.101:3303/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d44012a10 con 0x7f3d4c13b470 2023-01-25T02:49:05.357 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:05.357 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: got monmap 1 from mon.i (according to old e1) 2023-01-25T02:49:05.358 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.326+0000 7f3d49d9b700 10 monclient: dump: 2023-01-25T02:49:05.358 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-01-25T02:49:05.358 INFO:teuthology.misc.health.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:05.358 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:05.358 INFO:teuthology.misc.health.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:05.359 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:05.359 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:05.359 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:05.359 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:05.360 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:05.360 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:05.360 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:05.360 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:05.361 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:05.361 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:05.361 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:05.361 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:05.361 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 10 monclient: shutdown 2023-01-25T02:49:05.362 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:05.362 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 1 -- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3d4c13b470 msgr2=0x7f3d4c13d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:05.362 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 1 --2- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3d4c13b470 0x7f3d4c13d850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009dc0 tx=0x7f3d4400b330 comp rx=0 tx=0).stop 2023-01-25T02:49:05.362 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 1 -- 172.21.15.19:0/2100183427 shutdown_connections 2023-01-25T02:49:05.362 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 1 --2- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] conn(0x7f3d4c13b470 0x7f3d4c13d850 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.363 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 1 --2- 172.21.15.19:0/2100183427 >> [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] conn(0x7f3d4c136d10 0x7f3d4c0a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.363 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 1 -- 172.21.15.19:0/2100183427 >> 172.21.15.19:0/2100183427 conn(0x7f3d4c0a5710 msgr2=0x7f3d4c0a5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:05.363 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 1 -- 172.21.15.19:0/2100183427 shutdown_connections 2023-01-25T02:49:05.363 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.327+0000 7f3d51107700 1 -- 172.21.15.19:0/2100183427 wait complete. 2023-01-25T02:49:05.364 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.328+0000 7f3d51107700 10 monclient: build_initial_monmap 2023-01-25T02:49:05.364 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.328+0000 7f3d51107700 10 monclient: monmap: 2023-01-25T02:49:05.364 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-01-25T02:49:05.364 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:49:05.364 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-01-25T02:49:05.365 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-01-25T02:49:05.365 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:05.365 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-01-25T02:49:05.365 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:49:05.366 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:49:05.366 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:05.366 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:05.366 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:05.366 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:05.367 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:05.367 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:05.367 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:05.367 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:05.368 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.328+0000 7f3d51107700 1 Processor -- start 2023-01-25T02:49:05.368 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.328+0000 7f3d51107700 1 -- start start 2023-01-25T02:49:05.368 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.328+0000 7f3d51107700 10 monclient: init 2023-01-25T02:49:05.369 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:05.369 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient: _add_conns ranks=[8,6,0,3,7,4,5,2,1] 2023-01-25T02:49:05.369 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3d4c136230 0x7f3d4c0a7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:05.369 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient(hunting): picked mon.noname-e con 0x7f3d4c136230 addr [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] 2023-01-25T02:49:05.369 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3d4c136d10 0x7f3d4c0a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:05.369 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient(hunting): picked mon.noname-d con 0x7f3d4c136d10 addr [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] 2023-01-25T02:49:05.370 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c13b470 0x7f3d4c0a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:05.370 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient(hunting): picked mon.noname-a con 0x7f3d4c13b470 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-01-25T02:49:05.370 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:05.370 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c137e90 con 0x7f3d4c13b470 2023-01-25T02:49:05.370 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:05.371 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 1 -- --> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c09ba70 con 0x7f3d4c136d10 2023-01-25T02:49:05.371 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:05.371 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 1 -- --> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0a11d0 con 0x7f3d4c136230 2023-01-25T02:49:05.371 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:05.371 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d51107700 10 monclient(hunting): authenticate will time out at 954.854142s 2023-01-25T02:49:05.372 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3d4c136230 0x7f3d4c0a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:05.372 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3d4c136d10 0x7f3d4c0a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:05.372 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c13b470 0x7f3d4c0a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:05.372 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3d4c136d10 0x7f3d4c0a76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3303/0 says I am v2:172.21.15.19:40328/0 (socket says 172.21.15.19:40328) 2023-01-25T02:49:05.373 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 1 -- 172.21.15.19:0/1572476669 learned_addr learned my addr 172.21.15.19:0/1572476669 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-01-25T02:49:05.373 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3d4c136230 0x7f3d4c0a7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3304/0 says I am v2:172.21.15.19:59012/0 (socket says 172.21.15.19:59012) 2023-01-25T02:49:05.373 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 10 monclient(hunting): get_auth_request con 0x7f3d4c136d10 auth_method 0 2023-01-25T02:49:05.373 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:05.373 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:05.374 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:05.374 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient(hunting): get_auth_request con 0x7f3d4c13b470 auth_method 0 2023-01-25T02:49:05.374 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:05.374 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:05.374 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:05.375 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 10 monclient(hunting): get_auth_request con 0x7f3d4c136230 auth_method 0 2023-01-25T02:49:05.375 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:05.375 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:05.375 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:05.375 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:05.375 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:05.376 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:05.376 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:05.376 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:05.376 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:05.377 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:05.377 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:05.377 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:05.377 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient(hunting): handle_auth_done global_id 14211 payload 824 2023-01-25T02:49:05.378 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 1 -- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3d4c136230 msgr2=0x7f3d4c0a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:05.378 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3d4c136230 0x7f3d4c0a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.378 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 1 -- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3d4c136d10 msgr2=0x7f3d4c0a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:05.378 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3d4c136d10 0x7f3d4c0a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.378 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient: _finish_hunting 0 2023-01-25T02:49:05.379 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 1 monclient: found mon.noname-a 2023-01-25T02:49:05.379 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:05.379 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.19:3300/0 2023-01-25T02:49:05.379 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 1 -- 172.21.15.19:0/1572476669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d44009a70 con 0x7f3d4c13b470 2023-01-25T02:49:05.379 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4a59c700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3d4c136d10 0x7f3d4c0a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:49:05.380 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4ad9d700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3d4c136230 0x7f3d4c0a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:49:05.380 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d4b59e700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c13b470 0x7f3d4c0a7bf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3d44016000 tx=0x7f3d44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:05.380 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d437fe700 1 -- 172.21.15.19:0/1572476669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d44012c20 con 0x7f3d4c13b470 2023-01-25T02:49:05.380 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.329+0000 7f3d437fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:05.381 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2023-01-25T02:49:05.381 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 10 monclient: dump: 2023-01-25T02:49:05.381 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-01-25T02:49:05.381 INFO:teuthology.misc.health.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:05.381 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:05.382 INFO:teuthology.misc.health.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:05.382 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:05.382 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:05.382 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:05.382 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:05.383 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:05.383 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:05.383 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:05.383 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:05.383 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:05.384 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:05.384 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:05.384 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:05.384 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 10 monclient: _finish_auth 0 2023-01-25T02:49:05.385 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 10 monclient: _check_auth_tickets 2023-01-25T02:49:05.385 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:05.385 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 1 -- 172.21.15.19:0/1572476669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d44012dc0 con 0x7f3d4c13b470 2023-01-25T02:49:05.385 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d51107700 5 monclient: authenticate success, global_id 14211 2023-01-25T02:49:05.386 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d51107700 10 monclient: _renew_subs 2023-01-25T02:49:05.386 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d51107700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-01-25T02:49:05.386 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c0a1a50 con 0x7f3d4c13b470 2023-01-25T02:49:05.386 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:05.386 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 1 -- 172.21.15.19:0/1572476669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d440044b0 con 0x7f3d4c13b470 2023-01-25T02:49:05.387 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:05.387 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 10 monclient: got monmap 1 from mon.a (according to old e1) 2023-01-25T02:49:05.387 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d4959a700 4 set_mon_vals no callback set 2023-01-25T02:49:05.387 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 10 monclient: dump: 2023-01-25T02:49:05.388 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-01-25T02:49:05.388 INFO:teuthology.misc.health.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:05.388 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:05.388 INFO:teuthology.misc.health.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:05.388 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:05.389 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:05.389 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:05.389 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:05.389 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:05.389 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:05.390 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:05.390 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:05.390 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:05.390 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:05.390 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:05.391 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:05.391 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d51107700 10 monclient: _renew_subs 2023-01-25T02:49:05.391 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d51107700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-01-25T02:49:05.391 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c0a1dc0 con 0x7f3d4c13b470 2023-01-25T02:49:05.391 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d437fe700 1 -- 172.21.15.19:0/1572476669 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 3) v1 ==== 42694+0+0 (secure 0 0 0) 0x7f3d44004040 con 0x7f3d4c13b470 2023-01-25T02:49:05.392 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.330+0000 7f3d51107700 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:05.392 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.331+0000 7f3d51107700 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:05.392 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.331+0000 7f3d51107700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-01-25T02:49:05.392 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.331+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c0a1770 con 0x7f3d4c13b470 2023-01-25T02:49:05.392 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.331+0000 7f3d437fe700 1 --2- 172.21.15.19:0/1572476669 >> v2:172.21.15.101:6800/101360 conn(0x7f3d2c036920 0x7f3d2c038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:05.393 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.331+0000 7f3d437fe700 1 -- 172.21.15.19:0/1572476669 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1611+0+0 (secure 0 0 0) 0x7f3d44048620 con 0x7f3d4c13b470 2023-01-25T02:49:05.393 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.331+0000 7f3d4ad9d700 1 --2- 172.21.15.19:0/1572476669 >> v2:172.21.15.101:6800/101360 conn(0x7f3d2c036920 0x7f3d2c038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:05.393 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.331+0000 7f3d4ad9d700 10 monclient: get_auth_request con 0x7f3d2c036920 auth_method 0 2023-01-25T02:49:05.393 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.331+0000 7f3d4ad9d700 1 --2- 172.21.15.19:0/1572476669 >> v2:172.21.15.101:6800/101360 conn(0x7f3d2c036920 0x7f3d2c038dd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c007fd0 tx=0x7f3d3c009040 comp rx=0 tx=0).ready entity=mgr.4115 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:05.394 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.336+0000 7f3d437fe700 1 -- 172.21.15.19:0/1572476669 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172632 (secure 0 0 0) 0x7f3d4402fd70 con 0x7f3d4c13b470 2023-01-25T02:49:05.394 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.336+0000 7f3d437fe700 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2023-01-25T02:49:05.394 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.336+0000 7f3d437fe700 10 monclient: _finish_command 1 = system:0 2023-01-25T02:49:05.531 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.531+0000 7f3d51107700 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2023-01-25T02:49:05.531 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.531+0000 7f3d51107700 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2023-01-25T02:49:05.531 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.531+0000 7f3d51107700 10 monclient: _send_mon_message to mon.a at v2:172.21.15.19:3300/0 2023-01-25T02:49:05.532 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.531+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d4c1381d0 con 0x7f3d4c13b470 2023-01-25T02:49:05.532 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.531+0000 7f3d437fe700 1 -- 172.21.15.19:0/1572476669 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) v1 ==== 74+0+5582 (secure 0 0 0) 0x7f3d440355e0 con 0x7f3d4c13b470 2023-01-25T02:49:05.532 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.531+0000 7f3d437fe700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2023-01-25T02:49:05.533 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.531+0000 7f3d437fe700 10 monclient: _finish_command 2 = system:0 2023-01-25T02:49:05.533 INFO:teuthology.misc.health.smithi019.stdout: 2023-01-25T02:49:05.534 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":9,"fsid":"079d300e-8dda-42d1-8c3c-224268975a31","created":"2023-01-25T02:48:52.592043+0000","modified":"2023-01-25T02:49:05.309084+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T02:49:04.305389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"635bff03-56e6-4b89-821e-97ea356fb68c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42002c5e-3e02-4cd5-a2c3-4cb5fc9e1560","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2235b6b3-ef48-49f8-925e-cbe7a454925c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1a0c3332-393c-4a3c-8955-882e0421a27e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8eee64a4-4f40-485d-8e81-8978cb8d9622","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c9a38bbd-d6c9-4369-99c5-9f484a2ba38b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T02:49:05.534 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 >> v2:172.21.15.101:6800/101360 conn(0x7f3d2c036920 msgr2=0x7f3d2c038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:05.535 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 --2- 172.21.15.19:0/1572476669 >> v2:172.21.15.101:6800/101360 conn(0x7f3d2c036920 0x7f3d2c038dd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c007fd0 tx=0x7f3d3c009040 comp rx=0 tx=0).stop 2023-01-25T02:49:05.535 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 10 monclient: shutdown 2023-01-25T02:49:05.535 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:05.535 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c13b470 msgr2=0x7f3d4c0a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:05.536 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c13b470 0x7f3d4c0a7bf0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3d44016000 tx=0x7f3d44004ad0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.536 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 shutdown_connections 2023-01-25T02:49:05.536 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 --2- 172.21.15.19:0/1572476669 >> v2:172.21.15.101:6800/101360 conn(0x7f3d2c036920 0x7f3d2c038dd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.536 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c13b470 0x7f3d4c0a7bf0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.536 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f3d4c136230 0x7f3d4c0a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.537 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 --2- 172.21.15.19:0/1572476669 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f3d4c136d10 0x7f3d4c0a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:05.537 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 >> 172.21.15.19:0/1572476669 conn(0x7f3d4c0a5710 msgr2=0x7f3d4c0a4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:05.537 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 shutdown_connections 2023-01-25T02:49:05.537 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:05.533+0000 7f3d51107700 1 -- 172.21.15.19:0/1572476669 wait complete. 2023-01-25T02:49:05.544 DEBUG:teuthology.misc:0 of 6 OSDs are up 2023-01-25T02:49:06.318 INFO:tasks.ceph.osd.4.smithi101.stderr:2023-01-25T02:49:06.317+0000 7f5ff839b700 -1 osd.4 0 waiting for initial osdmap 2023-01-25T02:49:06.319 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-01-25T02:49:06.318+0000 7f7ca5cdc700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T02:49:06.320 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-01-25T02:49:06.319+0000 7fca19bda700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T02:49:06.320 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-01-25T02:49:06.319+0000 7ff3f3401700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T02:49:06.321 INFO:tasks.ceph.osd.5.smithi101.stderr:2023-01-25T02:49:06.319+0000 7eff2fe46700 -1 osd.5 0 waiting for initial osdmap 2023-01-25T02:49:06.321 INFO:tasks.ceph.osd.4.smithi101.stderr:2023-01-25T02:49:06.320+0000 7f5ff3f7d700 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T02:49:06.321 INFO:tasks.ceph.osd.2.smithi019.stderr:2023-01-25T02:49:06.321+0000 7f7ca18be700 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T02:49:06.322 INFO:tasks.ceph.osd.5.smithi101.stderr:2023-01-25T02:49:06.321+0000 7eff2ba28700 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T02:49:06.322 INFO:tasks.ceph.osd.0.smithi019.stderr:2023-01-25T02:49:06.322+0000 7fca157bc700 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T02:49:06.323 INFO:tasks.ceph.osd.1.smithi019.stderr:2023-01-25T02:49:06.323+0000 7ff3eefe3700 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T02:49:06.328 INFO:tasks.ceph.osd.3.smithi101.stderr:2023-01-25T02:49:06.327+0000 7f8ba1c1c700 -1 osd.3 0 waiting for initial osdmap 2023-01-25T02:49:06.329 INFO:tasks.ceph.osd.3.smithi101.stderr:2023-01-25T02:49:06.328+0000 7f8b9d7fe700 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T02:49:12.149 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T02:49:12.275 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.274+0000 7f1d1c825700 10 monclient: get_monmap_and_config 2023-01-25T02:49:12.275 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.274+0000 7f1d1c825700 10 monclient: build_initial_monmap 2023-01-25T02:49:12.276 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.274+0000 7f1d1c825700 10 monclient: monmap: 2023-01-25T02:49:12.276 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-01-25T02:49:12.276 INFO:teuthology.misc.health.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:12.276 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-01-25T02:49:12.275258+0000 2023-01-25T02:49:12.277 INFO:teuthology.misc.health.smithi019.stderr:created 2023-01-25T02:49:12.275258+0000 2023-01-25T02:49:12.277 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:12.277 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:12.277 INFO:teuthology.misc.health.smithi019.stderr:0: v1:172.21.15.19:6789/0 mon.noname-a-legacy 2023-01-25T02:49:12.277 INFO:teuthology.misc.health.smithi019.stderr:1: v1:172.21.15.101:6789/0 mon.noname-f-legacy 2023-01-25T02:49:12.278 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:12.278 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:12.278 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:12.278 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:12.279 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:12.279 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:12.279 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:12.279 INFO:teuthology.misc.health.smithi019.stderr:9: v2:172.21.15.19:3300/0 mon.noname-a 2023-01-25T02:49:12.279 INFO:teuthology.misc.health.smithi019.stderr:10: v2:172.21.15.101:3300/0 mon.noname-f 2023-01-25T02:49:12.280 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:12.280 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 1 Processor -- start 2023-01-25T02:49:12.280 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 1 -- start start 2023-01-25T02:49:12.280 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient: init 2023-01-25T02:49:12.288 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:12.289 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient: _add_conns ranks=[6,3,4,2,10,0,9,8,1,5,7] 2023-01-25T02:49:12.289 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f1d18132680 0x7f1d18133010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:12.289 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient(hunting): picked mon.noname-d con 0x7f1d18132680 addr [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] 2023-01-25T02:49:12.289 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 1 --2- >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f1d1806d010 0x7f1d1806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:12.289 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient(hunting): picked mon.noname-g con 0x7f1d1806d010 addr [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] 2023-01-25T02:49:12.290 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d18135ba0 0x7f1d18136010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:12.290 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient(hunting): picked mon.noname-c con 0x7f1d18135ba0 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:49:12.290 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:12.290 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 1 -- --> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1806b8b0 con 0x7f1d1806d010 2023-01-25T02:49:12.290 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:12.291 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d18070160 con 0x7f1d18135ba0 2023-01-25T02:49:12.291 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:12.291 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 1 -- --> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] -- mon_getmap magic: 0 v1 -- 0x7f1d18133550 con 0x7f1d18132680 2023-01-25T02:49:12.291 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:12.291 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1c825700 10 monclient(hunting): authenticate will time out at 961.800802s 2023-01-25T02:49:12.292 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d16d9d700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d18135ba0 0x7f1d18136010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:12.292 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d1659c700 1 --2- >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f1d18132680 0x7f1d18133010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:12.292 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d16d9d700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d18135ba0 0x7f1d18136010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3302/0 says I am v2:172.21.15.19:34334/0 (socket says 172.21.15.19:34334) 2023-01-25T02:49:12.292 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d16d9d700 1 -- 172.21.15.19:0/2826440198 learned_addr learned my addr 172.21.15.19:0/2826440198 (peer_addr_for_me v2:172.21.15.19:0/0) 2023-01-25T02:49:12.292 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d16d9d700 10 monclient(hunting): get_auth_request con 0x7f1d18135ba0 auth_method 0 2023-01-25T02:49:12.293 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d16d9d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:12.293 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d16d9d700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:12.293 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.275+0000 7f1d16d9d700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:12.293 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1659c700 10 monclient(hunting): get_auth_request con 0x7f1d18132680 auth_method 0 2023-01-25T02:49:12.293 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1659c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:12.294 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1659c700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:12.294 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1659c700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:12.294 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1659c700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:12.294 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1659c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:12.294 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1659c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:12.295 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:12.295 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:12.295 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:12.295 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d15d9b700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f1d1806d010 0x7f1d1806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T02:49:12.295 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d15d9b700 10 monclient(hunting): get_auth_request con 0x7f1d1806d010 auth_method 0 2023-01-25T02:49:12.296 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d15d9b700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2023-01-25T02:49:12.296 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d15d9b700 10 monclient(hunting): _init_auth method 2 2023-01-25T02:49:12.296 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d15d9b700 10 monclient(hunting): _init_auth creating new auth 2023-01-25T02:49:12.296 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d15d9b700 10 monclient(hunting): handle_auth_reply_more payload 9 2023-01-25T02:49:12.296 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d15d9b700 10 monclient(hunting): handle_auth_reply_more payload_len 9 2023-01-25T02:49:12.297 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d15d9b700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2023-01-25T02:49:12.297 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 10 monclient(hunting): handle_auth_done global_id 4234 payload 274 2023-01-25T02:49:12.297 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 1 -- 172.21.15.19:0/2826440198 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f1d18132680 msgr2=0x7f1d18133010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:12.297 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f1d18132680 0x7f1d18133010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:12.298 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 1 -- 172.21.15.19:0/2826440198 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f1d1806d010 msgr2=0x7f1d1806d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-01-25T02:49:12.298 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f1d1806d010 0x7f1d1806d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:12.298 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 10 monclient: _finish_hunting 0 2023-01-25T02:49:12.298 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 1 monclient: found mon.noname-c 2023-01-25T02:49:12.298 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 20 monclient: _un_backoff reopen_interval_multipler now 1 2023-01-25T02:49:12.299 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.19:3302/0 2023-01-25T02:49:12.299 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 1 -- 172.21.15.19:0/2826440198 --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18061b00 con 0x7f1d18135ba0 2023-01-25T02:49:12.299 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1659c700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f1d18132680 0x7f1d18133010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-01-25T02:49:12.299 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d16d9d700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d18135ba0 0x7f1d18136010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009e80 tx=0x7f1d1000b330 comp rx=0 tx=0).ready entity=mon.4 client_cookie=1af7e900a7e53a9b server_cookie=0 in_seq=0 out_seq=0 2023-01-25T02:49:12.299 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 1 -- 172.21.15.19:0/2826440198 <== mon.4 v2:172.21.15.19:3302/0 1 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d1000e070 con 0x7f1d18135ba0 2023-01-25T02:49:12.300 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:12.300 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2023-01-25T02:49:12.300 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: dump: 2023-01-25T02:49:12.300 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-01-25T02:49:12.301 INFO:teuthology.misc.health.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:12.301 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:12.301 INFO:teuthology.misc.health.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:12.301 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:12.301 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:12.302 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:12.302 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:12.302 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:12.302 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:12.302 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:12.303 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:12.303 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:12.303 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:12.303 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:12.303 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:12.303 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: _finish_auth 0 2023-01-25T02:49:12.304 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: _check_auth_tickets 2023-01-25T02:49:12.304 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 20 monclient: _check_auth_rotating not needed by client.admin 2023-01-25T02:49:12.304 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1c825700 5 monclient: authenticate success, global_id 4234 2023-01-25T02:49:12.304 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 1 -- 172.21.15.19:0/2826440198 <== mon.4 v2:172.21.15.19:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1000bd80 con 0x7f1d18135ba0 2023-01-25T02:49:12.305 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1c825700 20 monclient: get_monmap_and_config waiting for monmap|config 2023-01-25T02:49:12.305 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: handle_config config(0 keys) v1 2023-01-25T02:49:12.305 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1c825700 10 monclient: get_monmap_and_config success 2023-01-25T02:49:12.305 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 1 -- 172.21.15.19:0/2826440198 <== mon.4 v2:172.21.15.19:3302/0 3 ==== mon_map magic: 0 v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1d10012860 con 0x7f1d18135ba0 2023-01-25T02:49:12.306 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1c825700 4 set_mon_vals no callback set 2023-01-25T02:49:12.306 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: handle_monmap mon_map magic: 0 v1 2023-01-25T02:49:12.306 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: got monmap 1 from mon.c (according to old e1) 2023-01-25T02:49:12.306 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.276+0000 7f1d1559a700 10 monclient: dump: 2023-01-25T02:49:12.306 INFO:teuthology.misc.health.smithi019.stderr:epoch 1 2023-01-25T02:49:12.307 INFO:teuthology.misc.health.smithi019.stderr:fsid 079d300e-8dda-42d1-8c3c-224268975a31 2023-01-25T02:49:12.307 INFO:teuthology.misc.health.smithi019.stderr:last_changed 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:12.307 INFO:teuthology.misc.health.smithi019.stderr:created 2023-01-25T02:48:36.905639+0000 2023-01-25T02:49:12.307 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 18 (reef) 2023-01-25T02:49:12.307 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 3 2023-01-25T02:49:12.308 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2023-01-25T02:49:12.308 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.f 2023-01-25T02:49:12.308 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.b 2023-01-25T02:49:12.308 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.g 2023-01-25T02:49:12.308 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.c 2023-01-25T02:49:12.309 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.h 2023-01-25T02:49:12.309 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.d 2023-01-25T02:49:12.309 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.i 2023-01-25T02:49:12.309 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.e 2023-01-25T02:49:12.309 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:12.309 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 10 monclient: shutdown 2023-01-25T02:49:12.310 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 20 monclient: shutdown discarding 0 pending message(s) 2023-01-25T02:49:12.310 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 -- 172.21.15.19:0/2826440198 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d18135ba0 msgr2=0x7f1d18136010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T02:49:12.310 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d18135ba0 0x7f1d18136010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009e80 tx=0x7f1d1000b330 comp rx=0 tx=0).stop 2023-01-25T02:49:12.310 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 -- 172.21.15.19:0/2826440198 shutdown_connections 2023-01-25T02:49:12.310 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] conn(0x7f1d18132680 0x7f1d18133010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:12.311 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d18135ba0 0x7f1d18136010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:12.311 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 --2- 172.21.15.19:0/2826440198 >> [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] conn(0x7f1d1806d010 0x7f1d1806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T02:49:12.311 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 -- 172.21.15.19:0/2826440198 >> 172.21.15.19:0/2826440198 conn(0x7f1d18069ef0 msgr2=0x7f1d1806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T02:49:12.311 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 -- 172.21.15.19:0/2826440198 shutdown_connections 2023-01-25T02:49:12.311 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.277+0000 7f1d1c825700 1 -- 172.21.15.19:0/2826440198 wait complete. 2023-01-25T02:49:12.311 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient: build_initial_monmap 2023-01-25T02:49:12.312 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient: monmap: 2023-01-25T02:49:12.312 INFO:teuthology.misc.health.smithi019.stderr:epoch 0 2023-01-25T02:49:12.312 INFO:teuthology.misc.health.smithi019.stderr:fsid 00000000-0000-0000-0000-000000000000 2023-01-25T02:49:12.312 INFO:teuthology.misc.health.smithi019.stderr:last_changed 0.000000 2023-01-25T02:49:12.312 INFO:teuthology.misc.health.smithi019.stderr:created 0.000000 2023-01-25T02:49:12.313 INFO:teuthology.misc.health.smithi019.stderr:min_mon_release 0 (unknown) 2023-01-25T02:49:12.313 INFO:teuthology.misc.health.smithi019.stderr:election_strategy: 1 2023-01-25T02:49:12.313 INFO:teuthology.misc.health.smithi019.stderr:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.noname-a 2023-01-25T02:49:12.313 INFO:teuthology.misc.health.smithi019.stderr:1: [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon.noname-f 2023-01-25T02:49:12.314 INFO:teuthology.misc.health.smithi019.stderr:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.noname-b 2023-01-25T02:49:12.314 INFO:teuthology.misc.health.smithi019.stderr:3: [v2:172.21.15.101:3301/0,v1:172.21.15.101:6790/0] mon.noname-g 2023-01-25T02:49:12.314 INFO:teuthology.misc.health.smithi019.stderr:4: [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] mon.noname-c 2023-01-25T02:49:12.314 INFO:teuthology.misc.health.smithi019.stderr:5: [v2:172.21.15.101:3302/0,v1:172.21.15.101:6791/0] mon.noname-h 2023-01-25T02:49:12.314 INFO:teuthology.misc.health.smithi019.stderr:6: [v2:172.21.15.19:3303/0,v1:172.21.15.19:6792/0] mon.noname-d 2023-01-25T02:49:12.315 INFO:teuthology.misc.health.smithi019.stderr:7: [v2:172.21.15.101:3303/0,v1:172.21.15.101:6792/0] mon.noname-i 2023-01-25T02:49:12.315 INFO:teuthology.misc.health.smithi019.stderr:8: [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] mon.noname-e 2023-01-25T02:49:12.315 INFO:teuthology.misc.health.smithi019.stderr: 2023-01-25T02:49:12.315 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 1 Processor -- start 2023-01-25T02:49:12.315 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 1 -- start start 2023-01-25T02:49:12.316 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient: init 2023-01-25T02:49:12.316 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient: _reopen_session rank -1 2023-01-25T02:49:12.316 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient: _add_conns ranks=[4,8,0,6,2,7,1,5,3] 2023-01-25T02:49:12.316 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d1806d010 0x7f1d181ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:12.316 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient(hunting): picked mon.noname-c con 0x7f1d1806d010 addr [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] 2023-01-25T02:49:12.317 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 1 --2- >> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] conn(0x7f1d18132680 0x7f1d1812f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:12.317 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient(hunting): picked mon.noname-e con 0x7f1d18132680 addr [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] 2023-01-25T02:49:12.317 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d18135ba0 0x7f1d1812f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T02:49:12.317 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient(hunting): picked mon.noname-a con 0x7f1d18135ba0 addr [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] 2023-01-25T02:49:12.317 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:12.318 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180a05c0 con 0x7f1d18135ba0 2023-01-25T02:49:12.318 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:12.318 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 1 -- --> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180a2840 con 0x7f1d1806d010 2023-01-25T02:49:12.318 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient(hunting): start opening mon connection 2023-01-25T02:49:12.318 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 1 -- --> [v2:172.21.15.19:3304/0,v1:172.21.15.19:6793/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1809e5d0 con 0x7f1d18132680 2023-01-25T02:49:12.319 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.278+0000 7f1d1c825700 10 monclient(hunting): _renew_subs 2023-01-25T02:49:12.319 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.279+0000 7f1d1c825700 10 monclient(hunting): authenticate will time out at 961.804007s 2023-01-25T02:49:12.319 INFO:teuthology.misc.health.smithi019.stderr:2023-01-25T02:49:12.279+0000 7f1d1659c700 1 --2- >> [v2:172.21.15.19:3302/0,v1:172.21.15.19:6791/0] conn(0x7f1d1806d010 0x7f1d181ca170 unkno