2022-11-29T17:54:01.805 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-29T17:54:01.807 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi --description rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/failover} --owner scheduled_yuriw@teuthology --verbose --archive /home/teuthworker/archive/yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi/7096909 -- /home/teuthworker/archive/yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi/7096909/orig.config.yaml 2022-11-29T17:54:01.844 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-29T17:54:01.914 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi/7096909 branch: wip-yuri3-testing-2022-11-28-0750-pacific description: rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/failover} email: ceph-qa@ceph.io first_in_suite: false job_id: '7096909' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 2 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri3-testing-2022-11-28-0750-pacific ceph: conf: client: debug client: 20 debug mgrc: 20 debug ms: 1 global: mon election default strategy: 3 mds: debug mgrc: 20 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mgr ttl cache expire seconds: 5 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug mgrc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 8dc57b5624d909c1714a82ba84ff24e49431ea76 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8dc57b5624d909c1714a82ba84ff24e49431ea76 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2022-11-28-0750-pacific sha1: 8dc57b5624d909c1714a82ba84ff24e49431ea76 owner: scheduled_yuriw@teuthology priority: 61 repo: https://github.com/ceph/ceph-ci.git roles: - - mgr.x - mon.a - mon.c - mds.a - mds.c - osd.0 - client.0 - - mgr.y - mgr.z - mon.b - mds.b - osd.1 - osd.2 - osd.3 - client.1 sha1: 8dc57b5624d909c1714a82ba84ff24e49431ea76 sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri3-testing-2022-11-28-0750-pacific suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8dc57b5624d909c1714a82ba84ff24e49431ea76/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8dc57b5624d909c1714a82ba84ff24e49431ea76 targets: smithi035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtXAqgNSk3fbyauaAQ087MhP5pRS0EkYRcTNIno42bLKmY/5pysgUhxeVeBieflZ9GyMQdZn31ljFuM7O3aS0s/t+1nXnY4XhcvDDOO2eTUJKi0S/vviK1siPe1mR/aT8aCt38YDM7Hndl/PoU57GJc2K/RCANSTfyESt4x7TvkPTDsH6G13FV8TNL3D+hXCI2BugeRrLfkYzHwG4BaBON196sOtq80yagvFtCziW95BYvjprs6+mC0t6Csaw+uXHlhCHuh+M2nVFo4vH5U+TX53778kJ2DGe4X6yLRAeb7EZizfPO3D++zbSvroobAKjJioLD9HmIJFUDS5JZmizjQEuSkLk0MhMOY/tW6xVMXWi78qywZmWEmaMeAunbqrKaZLmF6/kZFy2iytetJTwTmw4KaTNbGCzCx5nDkicHX1N6JxN4r0LhuRRXkSYlPnRslPttny/9phAXcxO62p2i7EIC8pSFabccsgBniB5BEyEqJk2NzvCOt6ReFVpCKOM= smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKIsYmsaWdEaM9muLVCDHUqCdhAlsCmeYHg9Gh/4/zAQDldxMPHcBn/PozbC7uWUuTpR0nw4DjBvBacCM02Ub18= tasks: - install: null - ceph: log-ignorelist: - overall HEALTH_ - \(MGR_DOWN\) - \(PG_ - replacing it with standby - No standby daemons available wait-for-scrub: false - cephfs_test_runner: modules: - tasks.mgr.test_failover teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-29_15:35:32 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.11998 2022-11-29T17:54:01.914 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8dc57b5624d909c1714a82ba84ff24e49431ea76/qa; will attempt to use it 2022-11-29T17:54:01.915 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8dc57b5624d909c1714a82ba84ff24e49431ea76/qa/tasks 2022-11-29T17:54:01.915 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-29T17:54:01.917 INFO:teuthology.task.internal:Checking packages... 2022-11-29T17:54:01.943 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8dc57b5624d909c1714a82ba84ff24e49431ea76' 2022-11-29T17:54:01.943 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-29T17:54:01.944 INFO:teuthology.packaging:ref: None 2022-11-29T17:54:01.944 INFO:teuthology.packaging:tag: None 2022-11-29T17:54:01.944 INFO:teuthology.packaging:branch: wip-yuri3-testing-2022-11-28-0750-pacific 2022-11-29T17:54:01.945 INFO:teuthology.packaging:sha1: 8dc57b5624d909c1714a82ba84ff24e49431ea76 2022-11-29T17:54:01.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri3-testing-2022-11-28-0750-pacific 2022-11-29T17:54:02.101 INFO:teuthology.task.internal:Found packages for ceph version 16.2.10-1291-g8dc57b56-1focal 2022-11-29T17:54:02.101 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-29T17:54:02.109 INFO:teuthology.task.internal:no buildpackages task found 2022-11-29T17:54:02.109 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-29T17:54:02.115 INFO:teuthology.task.internal:Saving configuration 2022-11-29T17:54:02.129 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-29T17:54:02.135 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-29T17:54:02.166 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi035.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi/7096909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-29 17:48:51.365337', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPnDz7TZ8lJPlvgeOvxz0jQhTxq6lWaZkdNr+wpy+sDN'} 2022-11-29T17:54:02.196 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi/7096909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-29 17:48:51.363855', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM+W15jSwPXu4w7qFgY+UXp+IL0a9UO+J3fuvsMO4m9Y'} 2022-11-29T17:54:02.196 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-29T17:54:02.211 INFO:teuthology.task.internal:roles: ubuntu@smithi035.front.sepia.ceph.com - ['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'] 2022-11-29T17:54:02.211 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1'] 2022-11-29T17:54:02.211 INFO:teuthology.run_tasks:Running task console_log... 2022-11-29T17:54:02.291 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faac2d1df28>, signals=[15]) 2022-11-29T17:54:02.293 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-29T17:54:02.306 INFO:teuthology.task.internal:Opening connections... 2022-11-29T17:54:02.307 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi035.front.sepia.ceph.com 2022-11-29T17:54:02.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-29T17:54:02.421 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2022-11-29T17:54:02.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-29T17:54:02.528 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-29T17:54:02.536 DEBUG:teuthology.orchestra.run.smithi035:> uname -m 2022-11-29T17:54:02.543 INFO:teuthology.orchestra.run.smithi035.stdout:x86_64 2022-11-29T17:54:02.544 DEBUG:teuthology.orchestra.run.smithi035:> cat /etc/os-release 2022-11-29T17:54:02.594 INFO:teuthology.orchestra.run.smithi035.stdout:NAME="Ubuntu" 2022-11-29T17:54:02.594 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-11-29T17:54:02.594 INFO:teuthology.orchestra.run.smithi035.stdout:ID=ubuntu 2022-11-29T17:54:02.594 INFO:teuthology.orchestra.run.smithi035.stdout:ID_LIKE=debian 2022-11-29T17:54:02.595 INFO:teuthology.orchestra.run.smithi035.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-11-29T17:54:02.595 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_ID="20.04" 2022-11-29T17:54:02.595 INFO:teuthology.orchestra.run.smithi035.stdout:HOME_URL="https://www.ubuntu.com/" 2022-11-29T17:54:02.596 INFO:teuthology.orchestra.run.smithi035.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-11-29T17:54:02.596 INFO:teuthology.orchestra.run.smithi035.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-11-29T17:54:02.596 INFO:teuthology.orchestra.run.smithi035.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-11-29T17:54:02.597 INFO:teuthology.orchestra.run.smithi035.stdout:VERSION_CODENAME=focal 2022-11-29T17:54:02.597 INFO:teuthology.orchestra.run.smithi035.stdout:UBUNTU_CODENAME=focal 2022-11-29T17:54:02.598 INFO:teuthology.lock.ops:Updating smithi035.front.sepia.ceph.com on lock server 2022-11-29T17:54:02.619 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2022-11-29T17:54:02.626 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2022-11-29T17:54:02.627 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2022-11-29T17:54:02.633 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Ubuntu" 2022-11-29T17:54:02.634 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="20.04.5 LTS (Focal Fossa)" 2022-11-29T17:54:02.634 INFO:teuthology.orchestra.run.smithi038.stdout:ID=ubuntu 2022-11-29T17:54:02.634 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE=debian 2022-11-29T17:54:02.635 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Ubuntu 20.04.5 LTS" 2022-11-29T17:54:02.635 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="20.04" 2022-11-29T17:54:02.635 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.ubuntu.com/" 2022-11-29T17:54:02.635 INFO:teuthology.orchestra.run.smithi038.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-11-29T17:54:02.635 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-11-29T17:54:02.636 INFO:teuthology.orchestra.run.smithi038.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-11-29T17:54:02.636 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_CODENAME=focal 2022-11-29T17:54:02.636 INFO:teuthology.orchestra.run.smithi038.stdout:UBUNTU_CODENAME=focal 2022-11-29T17:54:02.637 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2022-11-29T17:54:02.656 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-29T17:54:02.666 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-29T17:54:02.672 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-29T17:54:02.673 DEBUG:teuthology.orchestra.run.smithi035:> test '!' -e /home/ubuntu/cephtest 2022-11-29T17:54:02.676 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2022-11-29T17:54:02.682 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-29T17:54:02.689 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-29T17:54:02.690 DEBUG:teuthology.orchestra.run.smithi035:> test -z $(ls -A /var/lib/ceph) 2022-11-29T17:54:02.693 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2022-11-29T17:54:02.703 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-29T17:54:02.760 INFO:teuthology.run_tasks:Running task kernel... 2022-11-29T17:54:02.777 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-11-29T17:54:02.778 INFO:teuthology.task.kernel:config {'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-11-29T17:54:02.779 DEBUG:teuthology.orchestra.run.smithi035:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-29T17:54:02.780 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-29T17:54:02.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-29T17:54:02.788 DEBUG:teuthology.orchestra.run.smithi035:> uname -r 2022-11-29T17:54:02.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-29T17:54:02.789 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2022-11-29T17:54:02.796 INFO:teuthology.orchestra.run.smithi035.stdout:5.4.0-128-generic 2022-11-29T17:54:02.797 INFO:teuthology.task.kernel:Running kernel on smithi035: 5.4.0-128-generic 2022-11-29T17:54:02.798 DEBUG:teuthology.orchestra.run.smithi035:> sudo apt-get clean 2022-11-29T17:54:02.800 INFO:teuthology.orchestra.run.smithi038.stdout:5.4.0-128-generic 2022-11-29T17:54:02.800 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.4.0-128-generic 2022-11-29T17:54:02.801 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get clean 2022-11-29T17:54:02.945 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get update 2022-11-29T17:54:03.009 DEBUG:teuthology.orchestra.run.smithi035:> sudo apt-get update 2022-11-29T17:54:03.105 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-29T17:54:03.111 INFO:teuthology.orchestra.run.smithi038.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-11-29T17:54:03.174 INFO:teuthology.orchestra.run.smithi035.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-11-29T17:54:03.180 INFO:teuthology.orchestra.run.smithi038.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-11-29T17:54:03.211 INFO:teuthology.orchestra.run.smithi038.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2022-11-29T17:54:03.278 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-29T17:54:03.359 INFO:teuthology.orchestra.run.smithi035.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2022-11-29T17:54:03.483 INFO:teuthology.orchestra.run.smithi035.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [524 kB] 2022-11-29T17:54:03.486 INFO:teuthology.orchestra.run.smithi038.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [756 kB] 2022-11-29T17:54:03.564 INFO:teuthology.orchestra.run.smithi038.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,236 kB] 2022-11-29T17:54:03.565 INFO:teuthology.orchestra.run.smithi035.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,861 kB] 2022-11-29T17:54:03.603 INFO:teuthology.orchestra.run.smithi038.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [390 kB] 2022-11-29T17:54:03.607 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.1 kB] 2022-11-29T17:54:03.608 INFO:teuthology.orchestra.run.smithi038.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [27.8 kB] 2022-11-29T17:54:03.609 INFO:teuthology.orchestra.run.smithi038.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,424 kB] 2022-11-29T17:54:03.611 INFO:teuthology.orchestra.run.smithi035.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [306 kB] 2022-11-29T17:54:03.611 INFO:teuthology.orchestra.run.smithi035.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [11.4 kB] 2022-11-29T17:54:03.611 INFO:teuthology.orchestra.run.smithi035.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,332 kB] 2022-11-29T17:54:03.616 INFO:teuthology.orchestra.run.smithi035.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [26.5 kB] 2022-11-29T17:54:03.617 INFO:teuthology.orchestra.run.smithi035.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [188 kB] 2022-11-29T17:54:03.618 INFO:teuthology.orchestra.run.smithi035.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [640 B] 2022-11-29T17:54:03.618 INFO:teuthology.orchestra.run.smithi035.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [573 kB] 2022-11-29T17:54:03.619 INFO:teuthology.orchestra.run.smithi038.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [202 kB] 2022-11-29T17:54:03.620 INFO:teuthology.orchestra.run.smithi038.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [636 B] 2022-11-29T17:54:03.621 INFO:teuthology.orchestra.run.smithi038.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [704 kB] 2022-11-29T17:54:03.622 INFO:teuthology.orchestra.run.smithi035.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [773 kB] 2022-11-29T17:54:03.626 INFO:teuthology.orchestra.run.smithi038.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,004 kB] 2022-11-29T17:54:03.627 INFO:teuthology.orchestra.run.smithi035.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [149 kB] 2022-11-29T17:54:03.630 INFO:teuthology.orchestra.run.smithi035.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [16.8 kB] 2022-11-29T17:54:03.630 INFO:teuthology.orchestra.run.smithi035.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.2 kB] 2022-11-29T17:54:03.631 INFO:teuthology.orchestra.run.smithi035.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2022-11-29T17:54:03.634 INFO:teuthology.orchestra.run.smithi038.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [233 kB] 2022-11-29T17:54:03.636 INFO:teuthology.orchestra.run.smithi038.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [23.2 kB] 2022-11-29T17:54:03.640 INFO:teuthology.orchestra.run.smithi038.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,448 B] 2022-11-29T17:54:03.640 INFO:teuthology.orchestra.run.smithi038.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.4 kB] 2022-11-29T17:54:03.640 INFO:teuthology.orchestra.run.smithi038.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [592 B] 2022-11-29T17:54:03.640 INFO:teuthology.orchestra.run.smithi038.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2022-11-29T17:54:03.647 INFO:teuthology.orchestra.run.smithi035.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,400 B] 2022-11-29T17:54:03.647 INFO:teuthology.orchestra.run.smithi035.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [516 B] 2022-11-29T17:54:03.653 INFO:teuthology.orchestra.run.smithi038.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2022-11-29T17:54:03.654 INFO:teuthology.orchestra.run.smithi038.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2022-11-29T17:54:03.654 INFO:teuthology.orchestra.run.smithi038.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.5 kB] 2022-11-29T17:54:03.655 INFO:teuthology.orchestra.run.smithi038.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.0 kB] 2022-11-29T17:54:03.656 INFO:teuthology.orchestra.run.smithi038.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [16.0 kB] 2022-11-29T17:54:03.656 INFO:teuthology.orchestra.run.smithi038.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [860 B] 2022-11-29T17:54:03.709 INFO:teuthology.orchestra.run.smithi035.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2022-11-29T17:54:03.798 INFO:teuthology.orchestra.run.smithi038.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [1,861 kB] 2022-11-29T17:54:03.850 INFO:teuthology.orchestra.run.smithi035.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,236 kB] 2022-11-29T17:54:04.196 INFO:teuthology.orchestra.run.smithi035.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [756 kB] 2022-11-29T17:54:04.231 INFO:teuthology.orchestra.run.smithi035.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [390 kB] 2022-11-29T17:54:04.232 INFO:teuthology.orchestra.run.smithi038.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [524 kB] 2022-11-29T17:54:04.243 INFO:teuthology.orchestra.run.smithi035.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [16.1 kB] 2022-11-29T17:54:04.244 INFO:teuthology.orchestra.run.smithi035.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [1,424 kB] 2022-11-29T17:54:04.248 INFO:teuthology.orchestra.run.smithi038.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [306 kB] 2022-11-29T17:54:04.256 INFO:teuthology.orchestra.run.smithi038.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [11.4 kB] 2022-11-29T17:54:04.257 INFO:teuthology.orchestra.run.smithi038.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [26.5 kB] 2022-11-29T17:54:04.257 INFO:teuthology.orchestra.run.smithi038.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [1,332 kB] 2022-11-29T17:54:04.282 INFO:teuthology.orchestra.run.smithi035.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [27.8 kB] 2022-11-29T17:54:04.283 INFO:teuthology.orchestra.run.smithi035.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [202 kB] 2022-11-29T17:54:04.287 INFO:teuthology.orchestra.run.smithi035.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [636 B] 2022-11-29T17:54:04.288 INFO:teuthology.orchestra.run.smithi035.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,004 kB] 2022-11-29T17:54:04.319 INFO:teuthology.orchestra.run.smithi038.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [188 kB] 2022-11-29T17:54:04.320 INFO:teuthology.orchestra.run.smithi035.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [704 kB] 2022-11-29T17:54:04.325 INFO:teuthology.orchestra.run.smithi038.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [640 B] 2022-11-29T17:54:04.325 INFO:teuthology.orchestra.run.smithi038.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [573 kB] 2022-11-29T17:54:04.330 INFO:teuthology.orchestra.run.smithi035.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [233 kB] 2022-11-29T17:54:04.340 INFO:teuthology.orchestra.run.smithi038.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [773 kB] 2022-11-29T17:54:04.357 INFO:teuthology.orchestra.run.smithi035.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [23.2 kB] 2022-11-29T17:54:04.358 INFO:teuthology.orchestra.run.smithi035.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [24.4 kB] 2022-11-29T17:54:04.359 INFO:teuthology.orchestra.run.smithi035.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,448 B] 2022-11-29T17:54:04.363 INFO:teuthology.orchestra.run.smithi038.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [149 kB] 2022-11-29T17:54:04.366 INFO:teuthology.orchestra.run.smithi038.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [16.8 kB] 2022-11-29T17:54:04.409 INFO:teuthology.orchestra.run.smithi038.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [22.2 kB] 2022-11-29T17:54:04.410 INFO:teuthology.orchestra.run.smithi038.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2022-11-29T17:54:04.410 INFO:teuthology.orchestra.run.smithi038.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,400 B] 2022-11-29T17:54:04.410 INFO:teuthology.orchestra.run.smithi038.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [516 B] 2022-11-29T17:54:04.411 INFO:teuthology.orchestra.run.smithi035.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [592 B] 2022-11-29T17:54:04.412 INFO:teuthology.orchestra.run.smithi035.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2022-11-29T17:54:04.412 INFO:teuthology.orchestra.run.smithi035.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2022-11-29T17:54:04.413 INFO:teuthology.orchestra.run.smithi035.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2022-11-29T17:54:04.414 INFO:teuthology.orchestra.run.smithi035.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.5 kB] 2022-11-29T17:54:04.436 INFO:teuthology.orchestra.run.smithi035.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [24.0 kB] 2022-11-29T17:54:04.437 INFO:teuthology.orchestra.run.smithi035.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [16.0 kB] 2022-11-29T17:54:04.517 INFO:teuthology.orchestra.run.smithi035.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [860 B] 2022-11-29T17:54:08.733 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 13.3 MB in 2s (6,468 kB/s) 2022-11-29T17:54:08.785 INFO:teuthology.orchestra.run.smithi035.stdout:Fetched 13.3 MB in 2s (6,585 kB/s) 2022-11-29T17:54:10.084 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2022-11-29T17:54:10.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-11-29T17:54:10.147 INFO:teuthology.orchestra.run.smithi035.stdout:Reading package lists... 2022-11-29T17:54:10.168 DEBUG:teuthology.orchestra.run.smithi035:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-11-29T17:54:10.180 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2022-11-29T17:54:10.239 INFO:teuthology.orchestra.run.smithi035.stdout:Reading package lists... 2022-11-29T17:54:10.367 INFO:teuthology.orchestra.run.smithi038.stdout:Building dependency tree... 2022-11-29T17:54:10.368 INFO:teuthology.orchestra.run.smithi038.stdout:Reading state information... 2022-11-29T17:54:10.428 INFO:teuthology.orchestra.run.smithi035.stdout:Building dependency tree... 2022-11-29T17:54:10.429 INFO:teuthology.orchestra.run.smithi035.stdout:Reading state information... 2022-11-29T17:54:10.543 INFO:teuthology.orchestra.run.smithi038.stdout:The following additional packages will be installed: 2022-11-29T17:54:10.543 INFO:teuthology.orchestra.run.smithi038.stdout: linux-generic linux-headers-5.4.0-132 linux-headers-5.4.0-132-generic 2022-11-29T17:54:10.544 INFO:teuthology.orchestra.run.smithi038.stdout: linux-headers-generic linux-image-5.4.0-132-generic 2022-11-29T17:54:10.544 INFO:teuthology.orchestra.run.smithi038.stdout: linux-modules-5.4.0-132-generic linux-modules-extra-5.4.0-132-generic 2022-11-29T17:54:10.544 INFO:teuthology.orchestra.run.smithi038.stdout:Suggested packages: 2022-11-29T17:54:10.545 INFO:teuthology.orchestra.run.smithi038.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-11-29T17:54:10.545 INFO:teuthology.orchestra.run.smithi038.stdout:Recommended packages: 2022-11-29T17:54:10.545 INFO:teuthology.orchestra.run.smithi038.stdout: thermald 2022-11-29T17:54:10.596 INFO:teuthology.orchestra.run.smithi038.stdout:The following NEW packages will be installed: 2022-11-29T17:54:10.597 INFO:teuthology.orchestra.run.smithi038.stdout: linux-headers-5.4.0-132 linux-headers-5.4.0-132-generic 2022-11-29T17:54:10.597 INFO:teuthology.orchestra.run.smithi038.stdout: linux-image-5.4.0-132-generic linux-modules-5.4.0-132-generic 2022-11-29T17:54:10.597 INFO:teuthology.orchestra.run.smithi038.stdout: linux-modules-extra-5.4.0-132-generic 2022-11-29T17:54:10.598 INFO:teuthology.orchestra.run.smithi038.stdout:The following packages will be upgraded: 2022-11-29T17:54:10.598 INFO:teuthology.orchestra.run.smithi038.stdout: linux-generic linux-headers-generic linux-image-generic 2022-11-29T17:54:10.610 INFO:teuthology.orchestra.run.smithi035.stdout:The following additional packages will be installed: 2022-11-29T17:54:10.610 INFO:teuthology.orchestra.run.smithi035.stdout: linux-generic linux-headers-5.4.0-132 linux-headers-5.4.0-132-generic 2022-11-29T17:54:10.611 INFO:teuthology.orchestra.run.smithi035.stdout: linux-headers-generic linux-image-5.4.0-132-generic 2022-11-29T17:54:10.611 INFO:teuthology.orchestra.run.smithi035.stdout: linux-modules-5.4.0-132-generic linux-modules-extra-5.4.0-132-generic 2022-11-29T17:54:10.612 INFO:teuthology.orchestra.run.smithi035.stdout:Suggested packages: 2022-11-29T17:54:10.612 INFO:teuthology.orchestra.run.smithi035.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2022-11-29T17:54:10.612 INFO:teuthology.orchestra.run.smithi035.stdout:Recommended packages: 2022-11-29T17:54:10.613 INFO:teuthology.orchestra.run.smithi035.stdout: thermald 2022-11-29T17:54:10.649 INFO:teuthology.orchestra.run.smithi038.stdout:3 upgraded, 5 newly installed, 0 to remove and 49 not upgraded. 2022-11-29T17:54:10.650 INFO:teuthology.orchestra.run.smithi038.stdout:Need to get 77.1 MB of archives. 2022-11-29T17:54:10.650 INFO:teuthology.orchestra.run.smithi038.stdout:After this operation, 380 MB of additional disk space will be used. 2022-11-29T17:54:10.651 INFO:teuthology.orchestra.run.smithi038.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-132-generic amd64 5.4.0-132.148 [15.0 MB] 2022-11-29T17:54:10.669 INFO:teuthology.orchestra.run.smithi035.stdout:The following NEW packages will be installed: 2022-11-29T17:54:10.669 INFO:teuthology.orchestra.run.smithi035.stdout: linux-headers-5.4.0-132 linux-headers-5.4.0-132-generic 2022-11-29T17:54:10.670 INFO:teuthology.orchestra.run.smithi035.stdout: linux-image-5.4.0-132-generic linux-modules-5.4.0-132-generic 2022-11-29T17:54:10.670 INFO:teuthology.orchestra.run.smithi035.stdout: linux-modules-extra-5.4.0-132-generic 2022-11-29T17:54:10.671 INFO:teuthology.orchestra.run.smithi035.stdout:The following packages will be upgraded: 2022-11-29T17:54:10.672 INFO:teuthology.orchestra.run.smithi035.stdout: linux-generic linux-headers-generic linux-image-generic 2022-11-29T17:54:10.849 INFO:teuthology.orchestra.run.smithi035.stdout:3 upgraded, 5 newly installed, 0 to remove and 49 not upgraded. 2022-11-29T17:54:10.849 INFO:teuthology.orchestra.run.smithi035.stdout:Need to get 77.1 MB of archives. 2022-11-29T17:54:10.849 INFO:teuthology.orchestra.run.smithi035.stdout:After this operation, 380 MB of additional disk space will be used. 2022-11-29T17:54:10.849 INFO:teuthology.orchestra.run.smithi035.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-132-generic amd64 5.4.0-132.148 [15.0 MB] 2022-11-29T17:54:10.946 INFO:teuthology.orchestra.run.smithi038.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-132-generic amd64 5.4.0-132.148 [10.5 MB] 2022-11-29T17:54:11.038 INFO:teuthology.orchestra.run.smithi038.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-132-generic amd64 5.4.0-132.148 [39.2 MB] 2022-11-29T17:54:11.548 INFO:teuthology.orchestra.run.smithi038.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.132.132 [1,900 B] 2022-11-29T17:54:11.548 INFO:teuthology.orchestra.run.smithi038.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.132.132 [2,516 B] 2022-11-29T17:54:11.549 INFO:teuthology.orchestra.run.smithi038.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-132 all 5.4.0-132.148 [11.0 MB] 2022-11-29T17:54:11.644 INFO:teuthology.orchestra.run.smithi038.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-132-generic amd64 5.4.0-132.148 [1,402 kB] 2022-11-29T17:54:11.655 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.132.132 [2,384 B] 2022-11-29T17:54:11.972 INFO:teuthology.orchestra.run.smithi035.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-132-generic amd64 5.4.0-132.148 [10.5 MB] 2022-11-29T17:54:12.101 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 77.1 MB in 1s (73.4 MB/s) 2022-11-29T17:54:12.238 INFO:teuthology.orchestra.run.smithi035.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-132-generic amd64 5.4.0-132.148 [39.2 MB] 2022-11-29T17:54:12.296 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-modules-5.4.0-132-generic. 2022-11-29T17:54:13.495 INFO:teuthology.orchestra.run.smithi035.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.132.132 [1,900 B] 2022-11-29T17:54:13.495 INFO:teuthology.orchestra.run.smithi035.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.132.132 [2,516 B] 2022-11-29T17:54:13.496 INFO:teuthology.orchestra.run.smithi035.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-132 all 5.4.0-132.148 [11.0 MB] 2022-11-29T17:54:13.607 INFO:teuthology.orchestra.run.smithi038.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102417 files and directories currently installed.) 2022-11-29T17:54:13.612 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../0-linux-modules-5.4.0-132-generic_5.4.0-132.148_amd64.deb ... 2022-11-29T17:54:13.709 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-modules-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:13.849 INFO:teuthology.orchestra.run.smithi035.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-132-generic amd64 5.4.0-132.148 [1,402 kB] 2022-11-29T17:54:13.867 INFO:teuthology.orchestra.run.smithi035.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.132.132 [2,384 B] 2022-11-29T17:54:14.324 INFO:teuthology.orchestra.run.smithi035.stdout:Fetched 77.1 MB in 3s (24.2 MB/s) 2022-11-29T17:54:14.519 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package linux-modules-5.4.0-132-generic. 2022-11-29T17:54:15.626 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-image-5.4.0-132-generic. 2022-11-29T17:54:15.642 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../1-linux-image-5.4.0-132-generic_5.4.0-132.148_amd64.deb ... 2022-11-29T17:54:15.743 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-image-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:15.770 INFO:teuthology.orchestra.run.smithi035.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102417 files and directories currently installed.) 2022-11-29T17:54:15.775 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../0-linux-modules-5.4.0-132-generic_5.4.0-132.148_amd64.deb ... 2022-11-29T17:54:15.866 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking linux-modules-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:16.862 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-132-generic. 2022-11-29T17:54:16.875 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-132-generic_5.4.0-132.148_amd64.deb ... 2022-11-29T17:54:16.937 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-modules-extra-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:17.768 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package linux-image-5.4.0-132-generic. 2022-11-29T17:54:17.784 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../1-linux-image-5.4.0-132-generic_5.4.0-132.148_amd64.deb ... 2022-11-29T17:54:17.869 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking linux-image-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:19.013 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-132-generic. 2022-11-29T17:54:19.026 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-132-generic_5.4.0-132.148_amd64.deb ... 2022-11-29T17:54:19.054 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking linux-modules-extra-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:21.699 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../3-linux-generic_5.4.0.132.132_amd64.deb ... 2022-11-29T17:54:21.850 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-generic (5.4.0.132.132) over (5.4.0.128.129) ... 2022-11-29T17:54:22.142 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.132.132_amd64.deb ... 2022-11-29T17:54:22.271 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-image-generic (5.4.0.132.132) over (5.4.0.128.129) ... 2022-11-29T17:54:22.632 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-headers-5.4.0-132. 2022-11-29T17:54:22.646 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../5-linux-headers-5.4.0-132_5.4.0-132.148_all.deb ... 2022-11-29T17:54:22.690 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-headers-5.4.0-132 (5.4.0-132.148) ... 2022-11-29T17:54:23.794 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../3-linux-generic_5.4.0.132.132_amd64.deb ... 2022-11-29T17:54:23.946 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking linux-generic (5.4.0.132.132) over (5.4.0.128.129) ... 2022-11-29T17:54:24.296 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.132.132_amd64.deb ... 2022-11-29T17:54:24.425 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking linux-image-generic (5.4.0.132.132) over (5.4.0.128.129) ... 2022-11-29T17:54:24.770 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package linux-headers-5.4.0-132. 2022-11-29T17:54:24.787 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../5-linux-headers-5.4.0-132_5.4.0-132.148_all.deb ... 2022-11-29T17:54:24.820 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking linux-headers-5.4.0-132 (5.4.0-132.148) ... 2022-11-29T17:54:28.898 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package linux-headers-5.4.0-132-generic. 2022-11-29T17:54:28.917 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../6-linux-headers-5.4.0-132-generic_5.4.0-132.148_amd64.deb ... 2022-11-29T17:54:28.956 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-headers-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:30.264 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package linux-headers-5.4.0-132-generic. 2022-11-29T17:54:30.285 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../6-linux-headers-5.4.0-132-generic_5.4.0-132.148_amd64.deb ... 2022-11-29T17:54:30.330 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking linux-headers-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:32.058 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.132.132_amd64.deb ... 2022-11-29T17:54:32.220 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking linux-headers-generic (5.4.0.132.132) over (5.4.0.128.129) ... 2022-11-29T17:54:32.562 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-headers-5.4.0-132 (5.4.0-132.148) ... 2022-11-29T17:54:32.683 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-headers-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:32.818 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-headers-generic (5.4.0.132.132) ... 2022-11-29T17:54:32.969 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-modules-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:33.808 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.132.132_amd64.deb ... 2022-11-29T17:54:33.993 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking linux-headers-generic (5.4.0.132.132) over (5.4.0.128.129) ... 2022-11-29T17:54:34.212 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-image-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:34.309 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up linux-headers-5.4.0-132 (5.4.0-132.148) ... 2022-11-29T17:54:34.439 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up linux-headers-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:34.557 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up linux-headers-generic (5.4.0.132.132) ... 2022-11-29T17:54:34.708 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up linux-modules-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:35.365 INFO:teuthology.orchestra.run.smithi038.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-132-generic 2022-11-29T17:54:35.365 INFO:teuthology.orchestra.run.smithi038.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-132-generic 2022-11-29T17:54:35.498 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-modules-extra-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:36.064 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up linux-image-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:37.274 INFO:teuthology.orchestra.run.smithi035.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-132-generic 2022-11-29T17:54:37.274 INFO:teuthology.orchestra.run.smithi035.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-132-generic 2022-11-29T17:54:37.412 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up linux-modules-extra-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:37.453 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-image-generic (5.4.0.132.132) ... 2022-11-29T17:54:37.668 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up linux-generic (5.4.0.132.132) ... 2022-11-29T17:54:37.769 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for linux-image-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:37.817 INFO:teuthology.orchestra.run.smithi038.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-11-29T17:54:37.817 INFO:teuthology.orchestra.run.smithi038.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-132-generic 2022-11-29T17:54:38.655 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up linux-image-generic (5.4.0.132.132) ... 2022-11-29T17:54:38.756 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up linux-generic (5.4.0.132.132) ... 2022-11-29T17:54:39.553 INFO:teuthology.orchestra.run.smithi035.stdout:Processing triggers for linux-image-5.4.0-132-generic (5.4.0-132.148) ... 2022-11-29T17:54:39.610 INFO:teuthology.orchestra.run.smithi035.stdout:/etc/kernel/postinst.d/initramfs-tools: 2022-11-29T17:54:39.610 INFO:teuthology.orchestra.run.smithi035.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-132-generic 2022-11-29T17:55:00.373 INFO:teuthology.orchestra.run.smithi038.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-11-29T17:55:00.374 INFO:teuthology.orchestra.run.smithi038.stdout:Sourcing file `/etc/default/grub' 2022-11-29T17:55:00.389 INFO:teuthology.orchestra.run.smithi038.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-11-29T17:55:00.417 INFO:teuthology.orchestra.run.smithi038.stdout:Generating grub configuration file ... 2022-11-29T17:55:00.812 INFO:teuthology.orchestra.run.smithi038.stdout:Found linux image: /boot/vmlinuz-5.4.0-132-generic 2022-11-29T17:55:00.842 INFO:teuthology.orchestra.run.smithi038.stdout:Found initrd image: /boot/initrd.img-5.4.0-132-generic 2022-11-29T17:55:01.291 INFO:teuthology.orchestra.run.smithi038.stdout:Found linux image: /boot/vmlinuz-5.4.0-128-generic 2022-11-29T17:55:01.309 INFO:teuthology.orchestra.run.smithi038.stdout:Found initrd image: /boot/initrd.img-5.4.0-128-generic 2022-11-29T17:55:02.226 INFO:teuthology.orchestra.run.smithi035.stdout:/etc/kernel/postinst.d/zz-update-grub: 2022-11-29T17:55:02.226 INFO:teuthology.orchestra.run.smithi035.stdout:Sourcing file `/etc/default/grub' 2022-11-29T17:55:02.229 INFO:teuthology.orchestra.run.smithi035.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-11-29T17:55:02.243 INFO:teuthology.orchestra.run.smithi035.stdout:Generating grub configuration file ... 2022-11-29T17:55:02.631 INFO:teuthology.orchestra.run.smithi035.stdout:Found linux image: /boot/vmlinuz-5.4.0-132-generic 2022-11-29T17:55:02.672 INFO:teuthology.orchestra.run.smithi035.stdout:Found initrd image: /boot/initrd.img-5.4.0-132-generic 2022-11-29T17:55:03.109 INFO:teuthology.orchestra.run.smithi038.stdout:done 2022-11-29T17:55:03.115 INFO:teuthology.orchestra.run.smithi035.stdout:Found linux image: /boot/vmlinuz-5.4.0-128-generic 2022-11-29T17:55:03.127 INFO:teuthology.orchestra.run.smithi035.stdout:Found initrd image: /boot/initrd.img-5.4.0-128-generic 2022-11-29T17:55:03.505 DEBUG:teuthology.orchestra.run.smithi038:> dpkg -s linux-image-generic 2022-11-29T17:55:03.537 INFO:teuthology.orchestra.run.smithi038.stdout:Package: linux-image-generic 2022-11-29T17:55:03.537 INFO:teuthology.orchestra.run.smithi038.stdout:Status: install ok installed 2022-11-29T17:55:03.537 INFO:teuthology.orchestra.run.smithi038.stdout:Priority: optional 2022-11-29T17:55:03.538 INFO:teuthology.orchestra.run.smithi038.stdout:Section: kernel 2022-11-29T17:55:03.538 INFO:teuthology.orchestra.run.smithi038.stdout:Installed-Size: 19 2022-11-29T17:55:03.538 INFO:teuthology.orchestra.run.smithi038.stdout:Maintainer: Ubuntu Kernel Team 2022-11-29T17:55:03.538 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: amd64 2022-11-29T17:55:03.538 INFO:teuthology.orchestra.run.smithi038.stdout:Source: linux-meta 2022-11-29T17:55:03.539 INFO:teuthology.orchestra.run.smithi038.stdout:Version: 5.4.0.132.132 2022-11-29T17:55:03.539 INFO:teuthology.orchestra.run.smithi038.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-11-29T17:55:03.539 INFO:teuthology.orchestra.run.smithi038.stdout:Depends: linux-image-5.4.0-132-generic, linux-modules-extra-5.4.0-132-generic, linux-firmware, intel-microcode, amd64-microcode 2022-11-29T17:55:03.539 INFO:teuthology.orchestra.run.smithi038.stdout:Recommends: thermald 2022-11-29T17:55:03.540 INFO:teuthology.orchestra.run.smithi038.stdout:Description: Generic Linux kernel image 2022-11-29T17:55:03.540 INFO:teuthology.orchestra.run.smithi038.stdout: This package will always depend on the latest generic kernel image 2022-11-29T17:55:03.540 INFO:teuthology.orchestra.run.smithi038.stdout: available. 2022-11-29T17:55:03.540 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-128-generic Expected: 5.4.0-132-generic 2022-11-29T17:55:03.541 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-11-29T17:55:03.541 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-11-29T17:55:03.542 INFO:teuthology.task.kernel:Installing distro kernel on mgr.y... 2022-11-29T17:55:03.542 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=None, version=distro) 2022-11-29T17:55:03.542 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get clean 2022-11-29T17:55:03.612 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get update 2022-11-29T17:55:03.777 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-29T17:55:03.784 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-11-29T17:55:03.801 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-11-29T17:55:03.864 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-11-29T17:55:04.947 INFO:teuthology.orchestra.run.smithi035.stdout:done 2022-11-29T17:55:05.303 DEBUG:teuthology.orchestra.run.smithi035:> dpkg -s linux-image-generic 2022-11-29T17:55:05.340 INFO:teuthology.orchestra.run.smithi035.stdout:Package: linux-image-generic 2022-11-29T17:55:05.340 INFO:teuthology.orchestra.run.smithi035.stdout:Status: install ok installed 2022-11-29T17:55:05.341 INFO:teuthology.orchestra.run.smithi035.stdout:Priority: optional 2022-11-29T17:55:05.341 INFO:teuthology.orchestra.run.smithi035.stdout:Section: kernel 2022-11-29T17:55:05.341 INFO:teuthology.orchestra.run.smithi035.stdout:Installed-Size: 19 2022-11-29T17:55:05.343 INFO:teuthology.orchestra.run.smithi035.stdout:Maintainer: Ubuntu Kernel Team 2022-11-29T17:55:05.344 INFO:teuthology.orchestra.run.smithi035.stdout:Architecture: amd64 2022-11-29T17:55:05.344 INFO:teuthology.orchestra.run.smithi035.stdout:Source: linux-meta 2022-11-29T17:55:05.344 INFO:teuthology.orchestra.run.smithi035.stdout:Version: 5.4.0.132.132 2022-11-29T17:55:05.345 INFO:teuthology.orchestra.run.smithi035.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-11-29T17:55:05.345 INFO:teuthology.orchestra.run.smithi035.stdout:Depends: linux-image-5.4.0-132-generic, linux-modules-extra-5.4.0-132-generic, linux-firmware, intel-microcode, amd64-microcode 2022-11-29T17:55:05.345 INFO:teuthology.orchestra.run.smithi035.stdout:Recommends: thermald 2022-11-29T17:55:05.346 INFO:teuthology.orchestra.run.smithi035.stdout:Description: Generic Linux kernel image 2022-11-29T17:55:05.346 INFO:teuthology.orchestra.run.smithi035.stdout: This package will always depend on the latest generic kernel image 2022-11-29T17:55:05.346 INFO:teuthology.orchestra.run.smithi035.stdout: available. 2022-11-29T17:55:05.347 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-128-generic Expected: 5.4.0-132-generic 2022-11-29T17:55:05.347 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2022-11-29T17:55:05.348 DEBUG:teuthology.task.kernel:src is distro, skipping download 2022-11-29T17:55:05.348 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2022-11-29T17:55:05.348 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi035.front.sepia.ceph.com, path=None, version=distro) 2022-11-29T17:55:05.349 DEBUG:teuthology.orchestra.run.smithi035:> sudo apt-get clean 2022-11-29T17:55:05.458 DEBUG:teuthology.orchestra.run.smithi035:> sudo apt-get update 2022-11-29T17:55:05.463 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2022-11-29T17:55:05.487 DEBUG:teuthology.orchestra.run.smithi038:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-11-29T17:55:05.556 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2022-11-29T17:55:05.589 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-29T17:55:05.590 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-11-29T17:55:05.593 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-11-29T17:55:05.611 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-11-29T17:55:05.743 INFO:teuthology.orchestra.run.smithi038.stdout:Building dependency tree... 2022-11-29T17:55:05.744 INFO:teuthology.orchestra.run.smithi038.stdout:Reading state information... 2022-11-29T17:55:05.957 INFO:teuthology.orchestra.run.smithi038.stdout:linux-image-generic is already the newest version (5.4.0.132.132). 2022-11-29T17:55:05.957 INFO:teuthology.orchestra.run.smithi038.stdout:0 upgraded, 0 newly installed, 0 to remove and 49 not upgraded. 2022-11-29T17:55:05.959 DEBUG:teuthology.orchestra.run.smithi038:> dpkg -s linux-image-generic 2022-11-29T17:55:05.983 INFO:teuthology.orchestra.run.smithi038.stdout:Package: linux-image-generic 2022-11-29T17:55:05.983 INFO:teuthology.orchestra.run.smithi038.stdout:Status: install ok installed 2022-11-29T17:55:05.984 INFO:teuthology.orchestra.run.smithi038.stdout:Priority: optional 2022-11-29T17:55:05.984 INFO:teuthology.orchestra.run.smithi038.stdout:Section: kernel 2022-11-29T17:55:05.984 INFO:teuthology.orchestra.run.smithi038.stdout:Installed-Size: 19 2022-11-29T17:55:05.984 INFO:teuthology.orchestra.run.smithi038.stdout:Maintainer: Ubuntu Kernel Team 2022-11-29T17:55:05.985 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: amd64 2022-11-29T17:55:05.985 INFO:teuthology.orchestra.run.smithi038.stdout:Source: linux-meta 2022-11-29T17:55:05.985 INFO:teuthology.orchestra.run.smithi038.stdout:Version: 5.4.0.132.132 2022-11-29T17:55:05.985 INFO:teuthology.orchestra.run.smithi038.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-11-29T17:55:05.985 INFO:teuthology.orchestra.run.smithi038.stdout:Depends: linux-image-5.4.0-132-generic, linux-modules-extra-5.4.0-132-generic, linux-firmware, intel-microcode, amd64-microcode 2022-11-29T17:55:05.986 INFO:teuthology.orchestra.run.smithi038.stdout:Recommends: thermald 2022-11-29T17:55:05.986 INFO:teuthology.orchestra.run.smithi038.stdout:Description: Generic Linux kernel image 2022-11-29T17:55:05.986 INFO:teuthology.orchestra.run.smithi038.stdout: This package will always depend on the latest generic kernel image 2022-11-29T17:55:05.986 INFO:teuthology.orchestra.run.smithi038.stdout: available. 2022-11-29T17:55:05.987 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2022-11-29T17:55:05.993 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.tzlFaQr8aU 2022-11-29T17:55:05.993 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub/grub.cfg /tmp/tmp.tzlFaQr8aU 2022-11-29T17:55:06.019 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.tzlFaQr8aU 2022-11-29T17:55:06.102 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.tzlFaQr8aU is 9KB 2022-11-29T17:55:06.119 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.tzlFaQr8aU 2022-11-29T17:55:06.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-11-29T17:55:06.188 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T17:55:06.188 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-11-29T17:55:06.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-11-29T17:55:06.224 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-132-generic 2022-11-29T17:55:06.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo update-grub 2022-11-29T17:55:07.247 INFO:teuthology.orchestra.run.smithi035.stdout:Reading package lists... 2022-11-29T17:55:07.271 DEBUG:teuthology.orchestra.run.smithi035:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2022-11-29T17:55:07.340 INFO:teuthology.orchestra.run.smithi035.stdout:Reading package lists... 2022-11-29T17:55:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:Sourcing file `/etc/default/grub' 2022-11-29T17:55:07.479 INFO:teuthology.orchestra.run.smithi038.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-11-29T17:55:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2022-11-29T17:55:07.508 INFO:teuthology.orchestra.run.smithi035.stdout:Building dependency tree... 2022-11-29T17:55:07.509 INFO:teuthology.orchestra.run.smithi035.stdout:Reading state information... 2022-11-29T17:55:07.700 INFO:teuthology.orchestra.run.smithi035.stdout:linux-image-generic is already the newest version (5.4.0.132.132). 2022-11-29T17:55:07.700 INFO:teuthology.orchestra.run.smithi035.stdout:0 upgraded, 0 newly installed, 0 to remove and 49 not upgraded. 2022-11-29T17:55:07.702 DEBUG:teuthology.orchestra.run.smithi035:> dpkg -s linux-image-generic 2022-11-29T17:55:07.722 INFO:teuthology.orchestra.run.smithi035.stdout:Package: linux-image-generic 2022-11-29T17:55:07.723 INFO:teuthology.orchestra.run.smithi035.stdout:Status: install ok installed 2022-11-29T17:55:07.723 INFO:teuthology.orchestra.run.smithi035.stdout:Priority: optional 2022-11-29T17:55:07.723 INFO:teuthology.orchestra.run.smithi035.stdout:Section: kernel 2022-11-29T17:55:07.724 INFO:teuthology.orchestra.run.smithi035.stdout:Installed-Size: 19 2022-11-29T17:55:07.724 INFO:teuthology.orchestra.run.smithi035.stdout:Maintainer: Ubuntu Kernel Team 2022-11-29T17:55:07.724 INFO:teuthology.orchestra.run.smithi035.stdout:Architecture: amd64 2022-11-29T17:55:07.724 INFO:teuthology.orchestra.run.smithi035.stdout:Source: linux-meta 2022-11-29T17:55:07.724 INFO:teuthology.orchestra.run.smithi035.stdout:Version: 5.4.0.132.132 2022-11-29T17:55:07.725 INFO:teuthology.orchestra.run.smithi035.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.14) 2022-11-29T17:55:07.725 INFO:teuthology.orchestra.run.smithi035.stdout:Depends: linux-image-5.4.0-132-generic, linux-modules-extra-5.4.0-132-generic, linux-firmware, intel-microcode, amd64-microcode 2022-11-29T17:55:07.725 INFO:teuthology.orchestra.run.smithi035.stdout:Recommends: thermald 2022-11-29T17:55:07.725 INFO:teuthology.orchestra.run.smithi035.stdout:Description: Generic Linux kernel image 2022-11-29T17:55:07.726 INFO:teuthology.orchestra.run.smithi035.stdout: This package will always depend on the latest generic kernel image 2022-11-29T17:55:07.726 INFO:teuthology.orchestra.run.smithi035.stdout: available. 2022-11-29T17:55:07.727 DEBUG:teuthology.orchestra.run.smithi035:> mktemp 2022-11-29T17:55:07.734 INFO:teuthology.orchestra.run.smithi035.stdout:/tmp/tmp.IaPQ0ikSUh 2022-11-29T17:55:07.734 DEBUG:teuthology.orchestra.run.smithi035:> sudo cp /boot/grub/grub.cfg /tmp/tmp.IaPQ0ikSUh 2022-11-29T17:55:07.769 INFO:teuthology.orchestra.run.smithi038.stderr:Found linux image: /boot/vmlinuz-5.4.0-132-generic 2022-11-29T17:55:07.781 INFO:teuthology.orchestra.run.smithi038.stderr:Found initrd image: /boot/initrd.img-5.4.0-132-generic 2022-11-29T17:55:07.791 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 0666 /tmp/tmp.IaPQ0ikSUh 2022-11-29T17:55:07.830 DEBUG:teuthology.orchestra.remote:smithi035:/tmp/tmp.IaPQ0ikSUh is 9KB 2022-11-29T17:55:07.884 DEBUG:teuthology.orchestra.run.smithi035:> rm -fr /tmp/tmp.IaPQ0ikSUh 2022-11-29T17:55:07.891 DEBUG:teuthology.orchestra.run.smithi035:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2022-11-29T17:55:07.905 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T17:55:07.906 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/grub.d/01_ceph_kernel 2022-11-29T17:55:07.906 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2022-11-29T17:55:07.934 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-132-generic 2022-11-29T17:55:07.934 DEBUG:teuthology.orchestra.run.smithi035:> sudo update-grub 2022-11-29T17:55:08.205 INFO:teuthology.orchestra.run.smithi038.stderr:Found linux image: /boot/vmlinuz-5.4.0-128-generic 2022-11-29T17:55:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:Found initrd image: /boot/initrd.img-5.4.0-128-generic 2022-11-29T17:55:08.916 INFO:teuthology.orchestra.run.smithi038.stderr:done 2022-11-29T17:55:08.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2022-11-29T17:55:09.203 INFO:teuthology.orchestra.run.smithi035.stderr:Sourcing file `/etc/default/grub' 2022-11-29T17:55:09.205 INFO:teuthology.orchestra.run.smithi035.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2022-11-29T17:55:09.209 INFO:teuthology.orchestra.run.smithi035.stderr:Generating grub configuration file ... 2022-11-29T17:55:09.457 INFO:teuthology.orchestra.run.smithi035.stderr:Found linux image: /boot/vmlinuz-5.4.0-132-generic 2022-11-29T17:55:09.469 INFO:teuthology.orchestra.run.smithi035.stderr:Found initrd image: /boot/initrd.img-5.4.0-132-generic 2022-11-29T17:55:09.812 INFO:teuthology.orchestra.run.smithi035.stderr:Found linux image: /boot/vmlinuz-5.4.0-128-generic 2022-11-29T17:55:09.823 INFO:teuthology.orchestra.run.smithi035.stderr:Found initrd image: /boot/initrd.img-5.4.0-128-generic 2022-11-29T17:55:10.538 INFO:teuthology.orchestra.run.smithi035.stderr:done 2022-11-29T17:55:10.540 DEBUG:teuthology.orchestra.run.smithi035:> sudo shutdown -r now 2022-11-29T17:55:38.947 INFO:teuthology.task.kernel:Checking client mgr.y for new kernel version... 2022-11-29T17:55:38.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2022-11-29T17:55:38.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-29T17:55:40.545 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2022-11-29T17:55:40.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com' 2022-11-29T17:55:40.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-29T17:55:57.375 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.38 2022-11-29T17:55:58.911 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.35 2022-11-29T17:56:27.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2022-11-29T17:56:27.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-29T17:56:28.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com' 2022-11-29T17:56:28.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-29T17:56:43.038 DEBUG:teuthology.orchestra.run.smithi038:> true 2022-11-29T17:56:43.863 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2022-11-29T17:56:43.864 INFO:teuthology.task.kernel:Checking kernel version of mgr.y, want "5.4.0-132-generic"... 2022-11-29T17:56:43.864 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2022-11-29T17:56:43.871 INFO:teuthology.orchestra.run.smithi038.stdout:5.4.0-132-generic 2022-11-29T17:56:43.872 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-132-generic vs 5.4.0-132-generic 2022-11-29T17:56:43.872 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-11-29T17:56:43.872 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-11-29T17:56:44.491 DEBUG:teuthology.orchestra.run.smithi035:> true 2022-11-29T17:56:44.873 INFO:teuthology.task.kernel:Enabling kdb on mgr.y... 2022-11-29T17:56:44.874 DEBUG:teuthology.orchestra.run.smithi038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-29T17:56:44.931 INFO:teuthology.orchestra.run.smithi038.stdout:ttyS1 2022-11-29T17:56:44.952 DEBUG:teuthology.parallel:result is None 2022-11-29T17:56:45.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi035.front.sepia.ceph.com' 2022-11-29T17:56:45.033 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.4.0-132-generic"... 2022-11-29T17:56:45.033 DEBUG:teuthology.orchestra.run.smithi035:> uname -r 2022-11-29T17:56:45.040 INFO:teuthology.orchestra.run.smithi035.stdout:5.4.0-132-generic 2022-11-29T17:56:45.040 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-132-generic vs 5.4.0-132-generic 2022-11-29T17:56:45.041 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2022-11-29T17:56:45.041 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2022-11-29T17:56:46.042 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2022-11-29T17:56:46.043 DEBUG:teuthology.orchestra.run.smithi035:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-29T17:56:46.134 INFO:teuthology.orchestra.run.smithi035.stdout:ttyS1 2022-11-29T17:56:46.164 DEBUG:teuthology.parallel:result is None 2022-11-29T17:56:46.165 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-29T17:56:46.177 INFO:teuthology.task.internal:Creating test directory... 2022-11-29T17:56:46.178 DEBUG:teuthology.orchestra.run.smithi035:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-29T17:56:46.182 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-29T17:56:46.191 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-29T17:56:46.202 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-29T17:56:46.214 INFO:teuthology.task.internal:Creating archive directory... 2022-11-29T17:56:46.214 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-29T17:56:46.235 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-29T17:56:46.262 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-29T17:56:46.270 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-29T17:56:46.270 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-29T17:56:46.273 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-29T17:56:46.293 INFO:teuthology.orchestra.run.smithi035.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-29T17:56:46.296 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-29T17:56:46.304 INFO:teuthology.orchestra.run.smithi035.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-29T17:56:46.307 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-29T17:56:46.309 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-29T17:56:46.316 INFO:teuthology.task.internal:Configuring sudo... 2022-11-29T17:56:46.317 DEBUG:teuthology.orchestra.run.smithi035:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-29T17:56:46.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-29T17:56:46.365 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-29T17:56:46.376 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-29T17:56:46.376 DEBUG:teuthology.orchestra.run.smithi035:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-29T17:56:46.383 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-29T17:56:46.391 DEBUG:teuthology.orchestra.run.smithi035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-29T17:56:46.399 DEBUG:teuthology.orchestra.run.smithi035:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-29T17:56:46.447 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T17:56:46.448 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-29T17:56:46.470 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-29T17:56:46.477 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-29T17:56:46.485 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T17:56:46.485 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-29T17:56:46.507 DEBUG:teuthology.orchestra.run.smithi035:> sudo service rsyslog restart 2022-11-29T17:56:46.510 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2022-11-29T17:56:46.603 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-29T17:56:46.612 INFO:teuthology.task.internal:Starting timer... 2022-11-29T17:56:46.613 INFO:teuthology.run_tasks:Running task pcp... 2022-11-29T17:56:46.624 INFO:teuthology.run_tasks:Running task selinux... 2022-11-29T17:56:46.633 DEBUG:teuthology.task.selinux:Excluding smithi035: OS 'ubuntu' does not support SELinux 2022-11-29T17:56:46.634 DEBUG:teuthology.task.selinux:Excluding smithi038: OS 'ubuntu' does not support SELinux 2022-11-29T17:56:46.634 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-29T17:56:46.634 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2022-11-29T17:56:46.635 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-29T17:56:46.635 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-29T17:56:46.645 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2022-11-29T17:56:46.659 INFO:teuthology.repo_utils:Fetching main from origin 2022-11-29T17:56:46.798 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-29T17:56:46.819 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-29T17:56:46.821 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi035.front.sepia.ceph.com,smithi038.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-29T18:01:23.321 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi035.front.sepia.ceph.com'), Remote(name='ubuntu@smithi038.front.sepia.ceph.com')] 2022-11-29T18:01:23.326 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi035.front.sepia.ceph.com' 2022-11-29T18:01:23.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi035.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-29T18:01:23.429 DEBUG:teuthology.orchestra.run.smithi035:> true 2022-11-29T18:01:23.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2022-11-29T18:01:23.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-29T18:01:23.660 DEBUG:teuthology.orchestra.run.smithi038:> true 2022-11-29T18:01:23.789 INFO:teuthology.run_tasks:Running task clock... 2022-11-29T18:01:23.802 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-29T18:01:23.803 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-29T18:01:23.806 DEBUG:teuthology.orchestra.run.smithi035:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-29T18:01:23.808 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-29T18:01:23.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-29T18:01:23.838 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-11-29T18:01:23.838 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: Command line: ntpd -gq 2022-11-29T18:01:23.839 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: proto: precision = 0.105 usec (-23) 2022-11-29T18:01:23.839 INFO:teuthology.orchestra.run.smithi035.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-11-29T18:01:23.840 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-11-29T18:01:23.840 INFO:teuthology.orchestra.run.smithi035.stderr:restrict ::: KOD does nothing without LIMITED. 2022-11-29T18:01:23.841 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: restrict ::: KOD does nothing without LIMITED. 2022-11-29T18:01:23.842 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: Listen and drop on 0 v6wildcard [::]:123 2022-11-29T18:01:23.842 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-11-29T18:01:23.842 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: Listen normally on 2 lo 127.0.0.1:123 2022-11-29T18:01:23.842 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: Listen normally on 3 ens1f0 172.21.15.35:123 2022-11-29T18:01:23.842 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: Listen normally on 4 lo [::1]:123 2022-11-29T18:01:23.843 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25f6%4]:123 2022-11-29T18:01:23.843 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:23 ntpd[17192]: Listening on routing socket on fd #22 for interface updates 2022-11-29T18:01:23.868 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: ntpd 4.2.8p12@1.3728-o (1): Starting 2022-11-29T18:01:23.868 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: Command line: ntpd -gq 2022-11-29T18:01:23.869 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: proto: precision = 0.273 usec (-22) 2022-11-29T18:01:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-11-29T18:01:23.870 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2022-11-29T18:01:23.870 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: restrict ::: KOD does nothing without LIMITED. 2022-11-29T18:01:23.871 INFO:teuthology.orchestra.run.smithi038.stderr:restrict ::: KOD does nothing without LIMITED. 2022-11-29T18:01:23.872 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: Listen and drop on 0 v6wildcard [::]:123 2022-11-29T18:01:23.872 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2022-11-29T18:01:23.873 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: Listen normally on 2 lo 127.0.0.1:123 2022-11-29T18:01:23.873 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: Listen normally on 3 ens1f0 172.21.15.38:123 2022-11-29T18:01:23.873 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: Listen normally on 4 lo [::1]:123 2022-11-29T18:01:23.873 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15fc%4]:123 2022-11-29T18:01:23.874 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:23 ntpd[17204]: Listening on routing socket on fd #22 for interface updates 2022-11-29T18:01:24.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:24 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:24.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:24 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:24.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:24 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:24.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:24 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:24.842 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:24 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:24.842 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:24 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:24.869 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:24 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:24.870 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:24 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:24.870 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:24 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:24.870 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:24 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:24.871 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:24 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:24.871 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:24 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:26.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:26 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:26.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:26 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:26.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:26 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:26.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:26 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:26.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:26 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:26.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:26 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:26.870 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:26.871 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:26.871 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:26.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:26.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:26.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:26.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:26.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:26.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:26.875 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:26.875 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:26.875 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:26 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:27.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:27 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:27.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:27 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:27.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:27 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:27.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:27 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:27.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:27 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:27.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:27 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:27 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:27 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.842 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.842 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.842 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.842 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:28.842 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:28.843 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.843 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.843 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:28.843 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:28 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:28.871 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:28.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:28.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:28.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:28.875 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:28.875 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:28.875 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:28.875 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:28 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:29.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:29 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:29.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:29 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:29.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:29 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:29.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:29 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:29.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:29 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:29.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:29 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:30.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.839 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.840 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:30.841 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:30.841 INFO:teuthology.orchestra.run.smithi035.stdout:29 Nov 18:01:30 ntpd[17192]: ntpd: time slew -0.000952 s 2022-11-29T18:01:30.842 INFO:teuthology.orchestra.run.smithi035.stdout:ntpd: time slew -0.000952s 2022-11-29T18:01:30.842 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-11-29T18:01:30.843 INFO:teuthology.orchestra.run.smithi035.stderr:29 Nov 18:01:30 ntpd[17192]: can't open /var/log/ntpstats/loopstats.20221129: Permission denied 2022-11-29T18:01:30.870 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.870 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.871 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.871 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.871 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:30.872 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:30.873 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:30.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2022-11-29T18:01:30.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/rawstats.20221129: Permission denied 2022-11-29T18:01:30.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2022-11-29T18:01:30.874 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/peerstats.20221129: Permission denied 2022-11-29T18:01:30.875 INFO:teuthology.orchestra.run.smithi038.stdout:29 Nov 18:01:30 ntpd[17204]: ntpd: time slew -0.002209 s 2022-11-29T18:01:30.875 INFO:teuthology.orchestra.run.smithi038.stdout:ntpd: time slew -0.002209s 2022-11-29T18:01:30.876 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2022-11-29T18:01:30.876 INFO:teuthology.orchestra.run.smithi038.stderr:29 Nov 18:01:30 ntpd[17204]: can't open /var/log/ntpstats/loopstats.20221129: Permission denied 2022-11-29T18:01:30.897 INFO:teuthology.orchestra.run.smithi035.stdout: remote refid st t when poll reach delay offset jitter 2022-11-29T18:01:30.898 INFO:teuthology.orchestra.run.smithi035.stdout:============================================================================== 2022-11-29T18:01:30.899 INFO:teuthology.orchestra.run.smithi035.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-11-29T18:01:30.900 INFO:teuthology.orchestra.run.smithi035.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-11-29T18:01:30.900 INFO:teuthology.orchestra.run.smithi035.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-11-29T18:01:30.900 INFO:teuthology.orchestra.run.smithi035.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-11-29T18:01:30.929 INFO:teuthology.orchestra.run.smithi038.stdout: remote refid st t when poll reach delay offset jitter 2022-11-29T18:01:30.929 INFO:teuthology.orchestra.run.smithi038.stdout:============================================================================== 2022-11-29T18:01:30.930 INFO:teuthology.orchestra.run.smithi038.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-11-29T18:01:30.930 INFO:teuthology.orchestra.run.smithi038.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-11-29T18:01:30.930 INFO:teuthology.orchestra.run.smithi038.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-11-29T18:01:30.930 INFO:teuthology.orchestra.run.smithi038.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2022-11-29T18:01:30.932 INFO:teuthology.run_tasks:Running task install... 2022-11-29T18:01:30.941 DEBUG:teuthology.task.install:project ceph 2022-11-29T18:01:30.942 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8dc57b5624d909c1714a82ba84ff24e49431ea76'}} 2022-11-29T18:01:30.942 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8dc57b5624d909c1714a82ba84ff24e49431ea76'} 2022-11-29T18:01:30.942 INFO:teuthology.task.install:Using flavor: default 2022-11-29T18:01:30.949 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-29T18:01:30.950 INFO:teuthology.task.install:extra packages: [] 2022-11-29T18:01:30.951 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2022-11-29T18:01:31.001 INFO:teuthology.orchestra.run.smithi035.stdout:Ubuntu 2022-11-29T18:01:31.007 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-11-29T18:01:31.008 DEBUG:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2022-11-29T18:01:31.008 DEBUG:teuthology.orchestra.run.smithi035:> sudo apt-key list | grep Ceph 2022-11-29T18:01:31.061 INFO:teuthology.orchestra.run.smithi038.stdout:Ubuntu 2022-11-29T18:01:31.067 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-11-29T18:01:31.068 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-key list | grep Ceph 2022-11-29T18:01:31.111 INFO:teuthology.orchestra.run.smithi035.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-11-29T18:01:31.170 INFO:teuthology.orchestra.run.smithi038.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2022-11-29T18:01:31.171 INFO:teuthology.orchestra.run.smithi035.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-11-29T18:01:31.171 INFO:teuthology.orchestra.run.smithi035.stdout:uid [ unknown] Ceph.com (release key) 2022-11-29T18:01:31.172 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2022-11-29T18:01:31.221 INFO:teuthology.orchestra.run.smithi035.stdout:Ubuntu 2022-11-29T18:01:31.226 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-11-29T18:01:31.227 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2022-11-29T18:01:31.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=8dc57b5624d909c1714a82ba84ff24e49431ea76 2022-11-29T18:01:31.234 INFO:teuthology.orchestra.run.smithi038.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2022-11-29T18:01:31.234 INFO:teuthology.orchestra.run.smithi038.stdout:uid [ unknown] Ceph.com (release key) 2022-11-29T18:01:31.235 DEBUG:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2022-11-29T18:01:31.287 INFO:teuthology.orchestra.run.smithi038.stdout:Ubuntu 2022-11-29T18:01:31.292 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-11-29T18:01:31.293 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2022-11-29T18:01:31.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=8dc57b5624d909c1714a82ba84ff24e49431ea76 2022-11-29T18:01:31.382 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default/ 2022-11-29T18:01:31.382 INFO:teuthology.task.install.deb:Package version is 16.2.10-1291-g8dc57b56-1focal 2022-11-29T18:01:31.442 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default/ 2022-11-29T18:01:31.442 INFO:teuthology.task.install.deb:Package version is 16.2.10-1291-g8dc57b56-1focal 2022-11-29T18:01:31.521 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:01:31.522 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-11-29T18:01:31.539 DEBUG:teuthology.orchestra.run.smithi035:> sudo apt-get update 2022-11-29T18:01:31.566 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:01:31.567 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2022-11-29T18:01:31.584 DEBUG:teuthology.orchestra.run.smithi038:> sudo apt-get update 2022-11-29T18:01:31.703 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-29T18:01:31.704 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-11-29T18:01:31.709 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-11-29T18:01:31.719 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2022-11-29T18:01:31.726 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2022-11-29T18:01:31.727 INFO:teuthology.orchestra.run.smithi035.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-11-29T18:01:31.743 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2022-11-29T18:01:31.759 INFO:teuthology.orchestra.run.smithi035.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal InRelease 2022-11-29T18:01:31.784 INFO:teuthology.orchestra.run.smithi035.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-11-29T18:01:31.784 INFO:teuthology.orchestra.run.smithi038.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal InRelease 2022-11-29T18:01:31.803 INFO:teuthology.orchestra.run.smithi038.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal Release [7,680 B] 2022-11-29T18:01:31.807 INFO:teuthology.orchestra.run.smithi035.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal Release.gpg 2022-11-29T18:01:31.828 INFO:teuthology.orchestra.run.smithi038.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal Release.gpg 2022-11-29T18:01:31.830 INFO:teuthology.orchestra.run.smithi035.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main i386 Packages [3,104 B] 2022-11-29T18:01:31.834 INFO:teuthology.orchestra.run.smithi038.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2022-11-29T18:01:31.851 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main i386 Packages [3,104 B] 2022-11-29T18:01:31.865 INFO:teuthology.orchestra.run.smithi035.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 Packages [17.2 kB] 2022-11-29T18:01:31.894 INFO:teuthology.orchestra.run.smithi038.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 Packages [17.2 kB] 2022-11-29T18:01:32.381 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 28.0 kB in 1s (43.9 kB/s) 2022-11-29T18:01:32.384 INFO:teuthology.orchestra.run.smithi035.stdout:Fetched 28.0 kB in 1s (41.6 kB/s) 2022-11-29T18:01:33.339 INFO:teuthology.orchestra.run.smithi035.stdout:Reading package lists... 2022-11-29T18:01:33.341 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2022-11-29T18:01:33.363 DEBUG:teuthology.orchestra.run.smithi035:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.10-1291-g8dc57b56-1focal cephadm=16.2.10-1291-g8dc57b56-1focal ceph-mds=16.2.10-1291-g8dc57b56-1focal ceph-mgr=16.2.10-1291-g8dc57b56-1focal ceph-common=16.2.10-1291-g8dc57b56-1focal ceph-fuse=16.2.10-1291-g8dc57b56-1focal ceph-test=16.2.10-1291-g8dc57b56-1focal radosgw=16.2.10-1291-g8dc57b56-1focal python3-rados=16.2.10-1291-g8dc57b56-1focal python3-rgw=16.2.10-1291-g8dc57b56-1focal python3-cephfs=16.2.10-1291-g8dc57b56-1focal python3-rbd=16.2.10-1291-g8dc57b56-1focal libcephfs2=16.2.10-1291-g8dc57b56-1focal libcephfs-dev=16.2.10-1291-g8dc57b56-1focal librados2=16.2.10-1291-g8dc57b56-1focal librbd1=16.2.10-1291-g8dc57b56-1focal rbd-fuse=16.2.10-1291-g8dc57b56-1focal 2022-11-29T18:01:33.367 DEBUG:teuthology.orchestra.run.smithi038:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.10-1291-g8dc57b56-1focal cephadm=16.2.10-1291-g8dc57b56-1focal ceph-mds=16.2.10-1291-g8dc57b56-1focal ceph-mgr=16.2.10-1291-g8dc57b56-1focal ceph-common=16.2.10-1291-g8dc57b56-1focal ceph-fuse=16.2.10-1291-g8dc57b56-1focal ceph-test=16.2.10-1291-g8dc57b56-1focal radosgw=16.2.10-1291-g8dc57b56-1focal python3-rados=16.2.10-1291-g8dc57b56-1focal python3-rgw=16.2.10-1291-g8dc57b56-1focal python3-cephfs=16.2.10-1291-g8dc57b56-1focal python3-rbd=16.2.10-1291-g8dc57b56-1focal libcephfs2=16.2.10-1291-g8dc57b56-1focal libcephfs-dev=16.2.10-1291-g8dc57b56-1focal librados2=16.2.10-1291-g8dc57b56-1focal librbd1=16.2.10-1291-g8dc57b56-1focal rbd-fuse=16.2.10-1291-g8dc57b56-1focal 2022-11-29T18:01:33.434 INFO:teuthology.orchestra.run.smithi035.stdout:Reading package lists... 2022-11-29T18:01:33.438 INFO:teuthology.orchestra.run.smithi038.stdout:Reading package lists... 2022-11-29T18:01:33.606 INFO:teuthology.orchestra.run.smithi038.stdout:Building dependency tree... 2022-11-29T18:01:33.607 INFO:teuthology.orchestra.run.smithi038.stdout:Reading state information... 2022-11-29T18:01:33.626 INFO:teuthology.orchestra.run.smithi035.stdout:Building dependency tree... 2022-11-29T18:01:33.626 INFO:teuthology.orchestra.run.smithi035.stdout:Reading state information... 2022-11-29T18:01:33.761 INFO:teuthology.orchestra.run.smithi038.stdout:The following packages were automatically installed and are no longer required: 2022-11-29T18:01:33.765 INFO:teuthology.orchestra.run.smithi038.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-11-29T18:01:33.765 INFO:teuthology.orchestra.run.smithi038.stdout:Use 'sudo apt autoremove' to remove them. 2022-11-29T18:01:33.766 INFO:teuthology.orchestra.run.smithi038.stdout:The following additional packages will be installed: 2022-11-29T18:01:33.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2022-11-29T18:01:33.766 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2022-11-29T18:01:33.767 INFO:teuthology.orchestra.run.smithi038.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-11-29T18:01:33.767 INFO:teuthology.orchestra.run.smithi038.stdout: liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-11-29T18:01:33.768 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libxslt1.1 nvme-cli 2022-11-29T18:01:33.768 INFO:teuthology.orchestra.run.smithi038.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2022-11-29T18:01:33.769 INFO:teuthology.orchestra.run.smithi038.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2022-11-29T18:01:33.769 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2022-11-29T18:01:33.770 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-11-29T18:01:33.770 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-11-29T18:01:33.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-11-29T18:01:33.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-11-29T18:01:33.771 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-11-29T18:01:33.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2022-11-29T18:01:33.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2022-11-29T18:01:33.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2022-11-29T18:01:33.772 INFO:teuthology.orchestra.run.smithi038.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2022-11-29T18:01:33.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2022-11-29T18:01:33.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket python3-webtest python3-werkzeug socat xmlstarlet 2022-11-29T18:01:33.773 INFO:teuthology.orchestra.run.smithi038.stdout:Suggested packages: 2022-11-29T18:01:33.773 INFO:teuthology.orchestra.run.smithi038.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-11-29T18:01:33.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-11-29T18:01:33.774 INFO:teuthology.orchestra.run.smithi038.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-11-29T18:01:33.774 INFO:teuthology.orchestra.run.smithi038.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-11-29T18:01:33.775 INFO:teuthology.orchestra.run.smithi038.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-11-29T18:01:33.775 INFO:teuthology.orchestra.run.smithi038.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-11-29T18:01:33.775 INFO:teuthology.orchestra.run.smithi038.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-11-29T18:01:33.775 INFO:teuthology.orchestra.run.smithi038.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-11-29T18:01:33.776 INFO:teuthology.orchestra.run.smithi038.stdout:Recommended packages: 2022-11-29T18:01:33.776 INFO:teuthology.orchestra.run.smithi038.stdout: btrfs-tools 2022-11-29T18:01:33.793 INFO:teuthology.orchestra.run.smithi035.stdout:The following packages were automatically installed and are no longer required: 2022-11-29T18:01:33.797 INFO:teuthology.orchestra.run.smithi035.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2022-11-29T18:01:33.797 INFO:teuthology.orchestra.run.smithi035.stdout:Use 'sudo apt autoremove' to remove them. 2022-11-29T18:01:33.798 INFO:teuthology.orchestra.run.smithi035.stdout:The following additional packages will be installed: 2022-11-29T18:01:33.799 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2022-11-29T18:01:33.799 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2022-11-29T18:01:33.800 INFO:teuthology.orchestra.run.smithi035.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2022-11-29T18:01:33.800 INFO:teuthology.orchestra.run.smithi035.stdout: liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 2022-11-29T18:01:33.800 INFO:teuthology.orchestra.run.smithi035.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libxslt1.1 nvme-cli 2022-11-29T18:01:33.800 INFO:teuthology.orchestra.run.smithi035.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2022-11-29T18:01:33.800 INFO:teuthology.orchestra.run.smithi035.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2022-11-29T18:01:33.801 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2022-11-29T18:01:33.801 INFO:teuthology.orchestra.run.smithi035.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2022-11-29T18:01:33.801 INFO:teuthology.orchestra.run.smithi035.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2022-11-29T18:01:33.801 INFO:teuthology.orchestra.run.smithi035.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2022-11-29T18:01:33.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2022-11-29T18:01:33.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2022-11-29T18:01:33.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2022-11-29T18:01:33.802 INFO:teuthology.orchestra.run.smithi035.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2022-11-29T18:01:33.803 INFO:teuthology.orchestra.run.smithi035.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2022-11-29T18:01:33.803 INFO:teuthology.orchestra.run.smithi035.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2022-11-29T18:01:33.803 INFO:teuthology.orchestra.run.smithi035.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2022-11-29T18:01:33.803 INFO:teuthology.orchestra.run.smithi035.stdout: python3-websocket python3-webtest python3-werkzeug socat xmlstarlet 2022-11-29T18:01:33.804 INFO:teuthology.orchestra.run.smithi035.stdout:Suggested packages: 2022-11-29T18:01:33.804 INFO:teuthology.orchestra.run.smithi035.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2022-11-29T18:01:33.804 INFO:teuthology.orchestra.run.smithi035.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2022-11-29T18:01:33.804 INFO:teuthology.orchestra.run.smithi035.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2022-11-29T18:01:33.804 INFO:teuthology.orchestra.run.smithi035.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2022-11-29T18:01:33.805 INFO:teuthology.orchestra.run.smithi035.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2022-11-29T18:01:33.805 INFO:teuthology.orchestra.run.smithi035.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2022-11-29T18:01:33.805 INFO:teuthology.orchestra.run.smithi035.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2022-11-29T18:01:33.806 INFO:teuthology.orchestra.run.smithi035.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2022-11-29T18:01:33.806 INFO:teuthology.orchestra.run.smithi035.stdout:Recommended packages: 2022-11-29T18:01:33.806 INFO:teuthology.orchestra.run.smithi035.stdout: btrfs-tools 2022-11-29T18:01:33.876 INFO:teuthology.orchestra.run.smithi038.stdout:The following NEW packages will be installed: 2022-11-29T18:01:33.877 INFO:teuthology.orchestra.run.smithi038.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-11-29T18:01:33.877 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2022-11-29T18:01:33.878 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2022-11-29T18:01:33.878 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 libjaeger libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-11-29T18:01:33.879 INFO:teuthology.orchestra.run.smithi038.stdout: libleveldb1d liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 2022-11-29T18:01:33.879 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libxslt1.1 nvme-cli 2022-11-29T18:01:33.879 INFO:teuthology.orchestra.run.smithi038.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2022-11-29T18:01:33.880 INFO:teuthology.orchestra.run.smithi038.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2022-11-29T18:01:33.880 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2022-11-29T18:01:33.881 INFO:teuthology.orchestra.run.smithi038.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-11-29T18:01:33.881 INFO:teuthology.orchestra.run.smithi038.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-11-29T18:01:33.882 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-11-29T18:01:33.882 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-11-29T18:01:33.882 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-11-29T18:01:33.883 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2022-11-29T18:01:33.883 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-11-29T18:01:33.884 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-11-29T18:01:33.884 INFO:teuthology.orchestra.run.smithi038.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-11-29T18:01:33.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-11-29T18:01:33.885 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-11-29T18:01:33.886 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-11-29T18:01:33.886 INFO:teuthology.orchestra.run.smithi038.stdout:The following packages will be upgraded: 2022-11-29T18:01:33.887 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 librbd1 2022-11-29T18:01:33.902 INFO:teuthology.orchestra.run.smithi035.stdout:The following NEW packages will be installed: 2022-11-29T18:01:33.903 INFO:teuthology.orchestra.run.smithi035.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2022-11-29T18:01:33.903 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2022-11-29T18:01:33.903 INFO:teuthology.orchestra.run.smithi035.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2022-11-29T18:01:33.903 INFO:teuthology.orchestra.run.smithi035.stdout: libcephfs2 libjaeger libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2022-11-29T18:01:33.904 INFO:teuthology.orchestra.run.smithi035.stdout: libleveldb1d liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 2022-11-29T18:01:33.906 INFO:teuthology.orchestra.run.smithi035.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libxslt1.1 nvme-cli 2022-11-29T18:01:33.906 INFO:teuthology.orchestra.run.smithi035.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2022-11-29T18:01:33.906 INFO:teuthology.orchestra.run.smithi035.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2022-11-29T18:01:33.906 INFO:teuthology.orchestra.run.smithi035.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2022-11-29T18:01:33.907 INFO:teuthology.orchestra.run.smithi035.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2022-11-29T18:01:33.907 INFO:teuthology.orchestra.run.smithi035.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2022-11-29T18:01:33.907 INFO:teuthology.orchestra.run.smithi035.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2022-11-29T18:01:33.907 INFO:teuthology.orchestra.run.smithi035.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2022-11-29T18:01:33.908 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2022-11-29T18:01:33.908 INFO:teuthology.orchestra.run.smithi035.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2022-11-29T18:01:33.908 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2022-11-29T18:01:33.908 INFO:teuthology.orchestra.run.smithi035.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2022-11-29T18:01:33.909 INFO:teuthology.orchestra.run.smithi035.stdout: python3-simplejson python3-singledispatch python3-sklearn 2022-11-29T18:01:33.909 INFO:teuthology.orchestra.run.smithi035.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2022-11-29T18:01:33.909 INFO:teuthology.orchestra.run.smithi035.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2022-11-29T18:01:33.909 INFO:teuthology.orchestra.run.smithi035.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2022-11-29T18:01:33.909 INFO:teuthology.orchestra.run.smithi035.stdout:The following packages will be upgraded: 2022-11-29T18:01:33.910 INFO:teuthology.orchestra.run.smithi035.stdout: librados2 librbd1 2022-11-29T18:01:33.938 INFO:teuthology.orchestra.run.smithi038.stdout:2 upgraded, 99 newly installed, 0 to remove and 46 not upgraded. 2022-11-29T18:01:33.938 INFO:teuthology.orchestra.run.smithi038.stdout:Need to get 174 MB of archives. 2022-11-29T18:01:33.938 INFO:teuthology.orchestra.run.smithi038.stdout:After this operation, 806 MB of additional disk space will be used. 2022-11-29T18:01:33.939 INFO:teuthology.orchestra.run.smithi038.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-11-29T18:01:33.967 INFO:teuthology.orchestra.run.smithi035.stdout:2 upgraded, 99 newly installed, 0 to remove and 46 not upgraded. 2022-11-29T18:01:33.967 INFO:teuthology.orchestra.run.smithi035.stdout:Need to get 174 MB of archives. 2022-11-29T18:01:33.967 INFO:teuthology.orchestra.run.smithi035.stdout:After this operation, 806 MB of additional disk space will be used. 2022-11-29T18:01:33.968 INFO:teuthology.orchestra.run.smithi035.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2022-11-29T18:01:34.012 INFO:teuthology.orchestra.run.smithi038.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libjaeger amd64 16.2.10-1291-g8dc57b56-1focal [3,892 B] 2022-11-29T18:01:34.024 INFO:teuthology.orchestra.run.smithi038.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-11-29T18:01:34.025 INFO:teuthology.orchestra.run.smithi038.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2022-11-29T18:01:34.033 INFO:teuthology.orchestra.run.smithi038.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-11-29T18:01:34.039 INFO:teuthology.orchestra.run.smithi035.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libjaeger amd64 16.2.10-1291-g8dc57b56-1focal [3,892 B] 2022-11-29T18:01:34.040 INFO:teuthology.orchestra.run.smithi038.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-11-29T18:01:34.054 INFO:teuthology.orchestra.run.smithi035.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2022-11-29T18:01:34.055 INFO:teuthology.orchestra.run.smithi035.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2022-11-29T18:01:34.058 INFO:teuthology.orchestra.run.smithi038.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.2.10-1291-g8dc57b56-1focal [3,376 kB] 2022-11-29T18:01:34.064 INFO:teuthology.orchestra.run.smithi038.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-11-29T18:01:34.064 INFO:teuthology.orchestra.run.smithi035.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2022-11-29T18:01:34.067 INFO:teuthology.orchestra.run.smithi038.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-11-29T18:01:34.069 INFO:teuthology.orchestra.run.smithi038.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-11-29T18:01:34.074 INFO:teuthology.orchestra.run.smithi035.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2022-11-29T18:01:34.077 INFO:teuthology.orchestra.run.smithi038.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-11-29T18:01:34.078 INFO:teuthology.orchestra.run.smithi038.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-11-29T18:01:34.078 INFO:teuthology.orchestra.run.smithi038.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-11-29T18:01:34.082 INFO:teuthology.orchestra.run.smithi038.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-11-29T18:01:34.082 INFO:teuthology.orchestra.run.smithi038.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-11-29T18:01:34.083 INFO:teuthology.orchestra.run.smithi038.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-11-29T18:01:34.085 INFO:teuthology.orchestra.run.smithi035.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.2.10-1291-g8dc57b56-1focal [3,376 kB] 2022-11-29T18:01:34.094 INFO:teuthology.orchestra.run.smithi035.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2022-11-29T18:01:34.096 INFO:teuthology.orchestra.run.smithi038.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2022-11-29T18:01:34.098 INFO:teuthology.orchestra.run.smithi038.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-11-29T18:01:34.098 INFO:teuthology.orchestra.run.smithi035.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2022-11-29T18:01:34.099 INFO:teuthology.orchestra.run.smithi038.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-11-29T18:01:34.099 INFO:teuthology.orchestra.run.smithi038.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-11-29T18:01:34.100 INFO:teuthology.orchestra.run.smithi035.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2022-11-29T18:01:34.101 INFO:teuthology.orchestra.run.smithi038.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-11-29T18:01:34.101 INFO:teuthology.orchestra.run.smithi038.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-11-29T18:01:34.106 INFO:teuthology.orchestra.run.smithi035.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2022-11-29T18:01:34.108 INFO:teuthology.orchestra.run.smithi035.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2022-11-29T18:01:34.109 INFO:teuthology.orchestra.run.smithi035.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2022-11-29T18:01:34.113 INFO:teuthology.orchestra.run.smithi035.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2022-11-29T18:01:34.113 INFO:teuthology.orchestra.run.smithi035.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2022-11-29T18:01:34.114 INFO:teuthology.orchestra.run.smithi038.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-11-29T18:01:34.114 INFO:teuthology.orchestra.run.smithi035.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2022-11-29T18:01:34.115 INFO:teuthology.orchestra.run.smithi038.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-11-29T18:01:34.115 INFO:teuthology.orchestra.run.smithi038.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-11-29T18:01:34.120 INFO:teuthology.orchestra.run.smithi038.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-11-29T18:01:34.126 INFO:teuthology.orchestra.run.smithi035.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2022-11-29T18:01:34.126 INFO:teuthology.orchestra.run.smithi035.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2022-11-29T18:01:34.127 INFO:teuthology.orchestra.run.smithi035.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2022-11-29T18:01:34.128 INFO:teuthology.orchestra.run.smithi035.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2022-11-29T18:01:34.128 INFO:teuthology.orchestra.run.smithi035.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2022-11-29T18:01:34.130 INFO:teuthology.orchestra.run.smithi038.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-11-29T18:01:34.130 INFO:teuthology.orchestra.run.smithi038.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-11-29T18:01:34.131 INFO:teuthology.orchestra.run.smithi038.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-11-29T18:01:34.133 INFO:teuthology.orchestra.run.smithi038.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-11-29T18:01:34.137 INFO:teuthology.orchestra.run.smithi038.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-11-29T18:01:34.139 INFO:teuthology.orchestra.run.smithi038.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-11-29T18:01:34.142 INFO:teuthology.orchestra.run.smithi035.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2022-11-29T18:01:34.144 INFO:teuthology.orchestra.run.smithi035.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2022-11-29T18:01:34.146 INFO:teuthology.orchestra.run.smithi035.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2022-11-29T18:01:34.146 INFO:teuthology.orchestra.run.smithi035.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2022-11-29T18:01:34.147 INFO:teuthology.orchestra.run.smithi038.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-11-29T18:01:34.148 INFO:teuthology.orchestra.run.smithi038.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-11-29T18:01:34.149 INFO:teuthology.orchestra.run.smithi038.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-11-29T18:01:34.149 INFO:teuthology.orchestra.run.smithi038.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-11-29T18:01:34.154 INFO:teuthology.orchestra.run.smithi035.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2022-11-29T18:01:34.159 INFO:teuthology.orchestra.run.smithi035.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2022-11-29T18:01:34.160 INFO:teuthology.orchestra.run.smithi035.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2022-11-29T18:01:34.161 INFO:teuthology.orchestra.run.smithi035.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2022-11-29T18:01:34.161 INFO:teuthology.orchestra.run.smithi035.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2022-11-29T18:01:34.162 INFO:teuthology.orchestra.run.smithi035.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2022-11-29T18:01:34.164 INFO:teuthology.orchestra.run.smithi038.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-11-29T18:01:34.164 INFO:teuthology.orchestra.run.smithi038.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-11-29T18:01:34.176 INFO:teuthology.orchestra.run.smithi035.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2022-11-29T18:01:34.177 INFO:teuthology.orchestra.run.smithi035.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2022-11-29T18:01:34.180 INFO:teuthology.orchestra.run.smithi035.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2022-11-29T18:01:34.182 INFO:teuthology.orchestra.run.smithi035.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2022-11-29T18:01:34.185 INFO:teuthology.orchestra.run.smithi035.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2022-11-29T18:01:34.193 INFO:teuthology.orchestra.run.smithi035.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2022-11-29T18:01:34.194 INFO:teuthology.orchestra.run.smithi035.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2022-11-29T18:01:34.245 INFO:teuthology.orchestra.run.smithi038.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-11-29T18:01:34.260 INFO:teuthology.orchestra.run.smithi038.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-11-29T18:01:34.261 INFO:teuthology.orchestra.run.smithi038.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-11-29T18:01:34.272 INFO:teuthology.orchestra.run.smithi038.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-11-29T18:01:34.273 INFO:teuthology.orchestra.run.smithi035.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2022-11-29T18:01:34.290 INFO:teuthology.orchestra.run.smithi035.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2022-11-29T18:01:34.291 INFO:teuthology.orchestra.run.smithi035.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2022-11-29T18:01:34.303 INFO:teuthology.orchestra.run.smithi035.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2022-11-29T18:01:34.334 INFO:teuthology.orchestra.run.smithi038.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-11-29T18:01:34.335 INFO:teuthology.orchestra.run.smithi038.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-11-29T18:01:34.337 INFO:teuthology.orchestra.run.smithi038.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-11-29T18:01:34.338 INFO:teuthology.orchestra.run.smithi038.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-11-29T18:01:34.340 INFO:teuthology.orchestra.run.smithi038.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-11-29T18:01:34.340 INFO:teuthology.orchestra.run.smithi038.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-11-29T18:01:34.342 INFO:teuthology.orchestra.run.smithi038.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-11-29T18:01:34.343 INFO:teuthology.orchestra.run.smithi038.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-11-29T18:01:34.345 INFO:teuthology.orchestra.run.smithi038.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.2.10-1291-g8dc57b56-1focal [3,379 kB] 2022-11-29T18:01:34.361 INFO:teuthology.orchestra.run.smithi038.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-11-29T18:01:34.363 INFO:teuthology.orchestra.run.smithi035.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2022-11-29T18:01:34.365 INFO:teuthology.orchestra.run.smithi038.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-11-29T18:01:34.366 INFO:teuthology.orchestra.run.smithi035.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2022-11-29T18:01:34.367 INFO:teuthology.orchestra.run.smithi035.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2022-11-29T18:01:34.368 INFO:teuthology.orchestra.run.smithi038.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-11-29T18:01:34.369 INFO:teuthology.orchestra.run.smithi035.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2022-11-29T18:01:34.369 INFO:teuthology.orchestra.run.smithi035.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2022-11-29T18:01:34.369 INFO:teuthology.orchestra.run.smithi035.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2022-11-29T18:01:34.370 INFO:teuthology.orchestra.run.smithi035.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2022-11-29T18:01:34.370 INFO:teuthology.orchestra.run.smithi035.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2022-11-29T18:01:34.371 INFO:teuthology.orchestra.run.smithi038.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-11-29T18:01:34.371 INFO:teuthology.orchestra.run.smithi038.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-11-29T18:01:34.373 INFO:teuthology.orchestra.run.smithi038.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libxslt1.1 amd64 1.1.34-4ubuntu0.20.04.1 [151 kB] 2022-11-29T18:01:34.378 INFO:teuthology.orchestra.run.smithi038.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-11-29T18:01:34.379 INFO:teuthology.orchestra.run.smithi038.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-11-29T18:01:34.380 INFO:teuthology.orchestra.run.smithi038.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-11-29T18:01:34.380 INFO:teuthology.orchestra.run.smithi038.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-11-29T18:01:34.388 INFO:teuthology.orchestra.run.smithi035.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2022-11-29T18:01:34.392 INFO:teuthology.orchestra.run.smithi035.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2022-11-29T18:01:34.394 INFO:teuthology.orchestra.run.smithi035.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2022-11-29T18:01:34.395 INFO:teuthology.orchestra.run.smithi038.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-11-29T18:01:34.395 INFO:teuthology.orchestra.run.smithi038.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-11-29T18:01:34.395 INFO:teuthology.orchestra.run.smithi038.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-11-29T18:01:34.396 INFO:teuthology.orchestra.run.smithi035.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2022-11-29T18:01:34.397 INFO:teuthology.orchestra.run.smithi035.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2022-11-29T18:01:34.401 INFO:teuthology.orchestra.run.smithi035.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libxslt1.1 amd64 1.1.34-4ubuntu0.20.04.1 [151 kB] 2022-11-29T18:01:34.402 INFO:teuthology.orchestra.run.smithi035.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2022-11-29T18:01:34.404 INFO:teuthology.orchestra.run.smithi035.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2022-11-29T18:01:34.407 INFO:teuthology.orchestra.run.smithi035.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2022-11-29T18:01:34.407 INFO:teuthology.orchestra.run.smithi035.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2022-11-29T18:01:34.412 INFO:teuthology.orchestra.run.smithi038.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-11-29T18:01:34.412 INFO:teuthology.orchestra.run.smithi038.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-11-29T18:01:34.412 INFO:teuthology.orchestra.run.smithi038.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-11-29T18:01:34.415 INFO:teuthology.orchestra.run.smithi038.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-11-29T18:01:34.416 INFO:teuthology.orchestra.run.smithi038.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-11-29T18:01:34.416 INFO:teuthology.orchestra.run.smithi038.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-11-29T18:01:34.416 INFO:teuthology.orchestra.run.smithi038.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-11-29T18:01:34.421 INFO:teuthology.orchestra.run.smithi035.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.2.10-1291-g8dc57b56-1focal [3,379 kB] 2022-11-29T18:01:34.421 INFO:teuthology.orchestra.run.smithi035.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2022-11-29T18:01:34.421 INFO:teuthology.orchestra.run.smithi035.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2022-11-29T18:01:34.422 INFO:teuthology.orchestra.run.smithi035.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2022-11-29T18:01:34.428 INFO:teuthology.orchestra.run.smithi038.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-11-29T18:01:34.430 INFO:teuthology.orchestra.run.smithi038.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-11-29T18:01:34.440 INFO:teuthology.orchestra.run.smithi035.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2022-11-29T18:01:34.448 INFO:teuthology.orchestra.run.smithi035.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2022-11-29T18:01:34.448 INFO:teuthology.orchestra.run.smithi035.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2022-11-29T18:01:34.449 INFO:teuthology.orchestra.run.smithi035.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2022-11-29T18:01:34.449 INFO:teuthology.orchestra.run.smithi035.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2022-11-29T18:01:34.449 INFO:teuthology.orchestra.run.smithi035.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2022-11-29T18:01:34.450 INFO:teuthology.orchestra.run.smithi035.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2022-11-29T18:01:34.455 INFO:teuthology.orchestra.run.smithi035.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2022-11-29T18:01:34.456 INFO:teuthology.orchestra.run.smithi035.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2022-11-29T18:01:34.613 INFO:teuthology.orchestra.run.smithi038.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.2.10-1291-g8dc57b56-1focal [725 kB] 2022-11-29T18:01:34.662 INFO:teuthology.orchestra.run.smithi038.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.2.10-1291-g8dc57b56-1focal [412 kB] 2022-11-29T18:01:34.675 INFO:teuthology.orchestra.run.smithi038.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.2.10-1291-g8dc57b56-1focal [22.1 kB] 2022-11-29T18:01:34.676 INFO:teuthology.orchestra.run.smithi038.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.2.10-1291-g8dc57b56-1focal [212 kB] 2022-11-29T18:01:34.689 INFO:teuthology.orchestra.run.smithi035.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.2.10-1291-g8dc57b56-1focal [725 kB] 2022-11-29T18:01:34.709 INFO:teuthology.orchestra.run.smithi038.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.2.10-1291-g8dc57b56-1focal [37.4 kB] 2022-11-29T18:01:34.710 INFO:teuthology.orchestra.run.smithi038.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.2.10-1291-g8dc57b56-1focal [384 kB] 2022-11-29T18:01:34.732 INFO:teuthology.orchestra.run.smithi038.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.2.10-1291-g8dc57b56-1focal [3,678 kB] 2022-11-29T18:01:34.749 INFO:teuthology.orchestra.run.smithi035.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.2.10-1291-g8dc57b56-1focal [412 kB] 2022-11-29T18:01:34.763 INFO:teuthology.orchestra.run.smithi035.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.2.10-1291-g8dc57b56-1focal [22.1 kB] 2022-11-29T18:01:34.765 INFO:teuthology.orchestra.run.smithi035.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.2.10-1291-g8dc57b56-1focal [212 kB] 2022-11-29T18:01:34.800 INFO:teuthology.orchestra.run.smithi035.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.2.10-1291-g8dc57b56-1focal [37.4 kB] 2022-11-29T18:01:34.801 INFO:teuthology.orchestra.run.smithi035.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.2.10-1291-g8dc57b56-1focal [384 kB] 2022-11-29T18:01:34.814 INFO:teuthology.orchestra.run.smithi035.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.2.10-1291-g8dc57b56-1focal [3,678 kB] 2022-11-29T18:01:34.987 INFO:teuthology.orchestra.run.smithi038.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.2.10-1291-g8dc57b56-1focal [116 kB] 2022-11-29T18:01:34.990 INFO:teuthology.orchestra.run.smithi038.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.2.10-1291-g8dc57b56-1focal [449 kB] 2022-11-29T18:01:35.038 INFO:teuthology.orchestra.run.smithi038.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.2.10-1291-g8dc57b56-1focal [22.5 MB] 2022-11-29T18:01:35.086 INFO:teuthology.orchestra.run.smithi035.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.2.10-1291-g8dc57b56-1focal [116 kB] 2022-11-29T18:01:35.089 INFO:teuthology.orchestra.run.smithi035.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.2.10-1291-g8dc57b56-1focal [449 kB] 2022-11-29T18:01:35.105 INFO:teuthology.orchestra.run.smithi035.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.2.10-1291-g8dc57b56-1focal [22.5 MB] 2022-11-29T18:01:36.624 INFO:teuthology.orchestra.run.smithi038.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.2.10-1291-g8dc57b56-1focal [5,909 kB] 2022-11-29T18:01:36.853 INFO:teuthology.orchestra.run.smithi035.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.2.10-1291-g8dc57b56-1focal [5,909 kB] 2022-11-29T18:01:37.000 INFO:teuthology.orchestra.run.smithi038.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.2.10-1291-g8dc57b56-1focal [203 kB] 2022-11-29T18:01:37.039 INFO:teuthology.orchestra.run.smithi038.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 16.2.10-1291-g8dc57b56-1focal [126 kB] 2022-11-29T18:01:37.102 INFO:teuthology.orchestra.run.smithi038.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.2.10-1291-g8dc57b56-1focal [1,529 kB] 2022-11-29T18:01:37.132 INFO:teuthology.orchestra.run.smithi038.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.2.10-1291-g8dc57b56-1focal [6,969 kB] 2022-11-29T18:01:37.256 INFO:teuthology.orchestra.run.smithi035.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.2.10-1291-g8dc57b56-1focal [203 kB] 2022-11-29T18:01:37.266 INFO:teuthology.orchestra.run.smithi035.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 16.2.10-1291-g8dc57b56-1focal [126 kB] 2022-11-29T18:01:37.270 INFO:teuthology.orchestra.run.smithi035.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.2.10-1291-g8dc57b56-1focal [1,529 kB] 2022-11-29T18:01:37.384 INFO:teuthology.orchestra.run.smithi035.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.2.10-1291-g8dc57b56-1focal [6,969 kB] 2022-11-29T18:01:37.654 INFO:teuthology.orchestra.run.smithi038.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.2.10-1291-g8dc57b56-1focal [26.2 MB] 2022-11-29T18:01:37.949 INFO:teuthology.orchestra.run.smithi035.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.2.10-1291-g8dc57b56-1focal [26.2 MB] 2022-11-29T18:01:39.700 INFO:teuthology.orchestra.run.smithi038.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.2.10-1291-g8dc57b56-1focal [3,980 B] 2022-11-29T18:01:39.702 INFO:teuthology.orchestra.run.smithi038.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.2.10-1291-g8dc57b56-1focal [830 kB] 2022-11-29T18:01:39.716 INFO:teuthology.orchestra.run.smithi038.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.2.10-1291-g8dc57b56-1focal [2,343 kB] 2022-11-29T18:01:39.851 INFO:teuthology.orchestra.run.smithi038.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.2.10-1291-g8dc57b56-1focal [74.7 kB] 2022-11-29T18:01:39.876 INFO:teuthology.orchestra.run.smithi038.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.2.10-1291-g8dc57b56-1focal [98.1 kB] 2022-11-29T18:01:39.879 INFO:teuthology.orchestra.run.smithi038.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.2.10-1291-g8dc57b56-1focal [2,819 kB] 2022-11-29T18:01:40.097 INFO:teuthology.orchestra.run.smithi038.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.2.10-1291-g8dc57b56-1focal [7,894 kB] 2022-11-29T18:01:40.146 INFO:teuthology.orchestra.run.smithi035.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.2.10-1291-g8dc57b56-1focal [3,980 B] 2022-11-29T18:01:40.147 INFO:teuthology.orchestra.run.smithi035.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.2.10-1291-g8dc57b56-1focal [830 kB] 2022-11-29T18:01:40.159 INFO:teuthology.orchestra.run.smithi035.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.2.10-1291-g8dc57b56-1focal [2,343 kB] 2022-11-29T18:01:40.302 INFO:teuthology.orchestra.run.smithi035.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.2.10-1291-g8dc57b56-1focal [74.7 kB] 2022-11-29T18:01:40.316 INFO:teuthology.orchestra.run.smithi035.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.2.10-1291-g8dc57b56-1focal [98.1 kB] 2022-11-29T18:01:40.319 INFO:teuthology.orchestra.run.smithi035.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.2.10-1291-g8dc57b56-1focal [2,819 kB] 2022-11-29T18:01:40.543 INFO:teuthology.orchestra.run.smithi035.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.2.10-1291-g8dc57b56-1focal [7,894 kB] 2022-11-29T18:01:40.632 INFO:teuthology.orchestra.run.smithi038.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.2.10-1291-g8dc57b56-1focal [4,096 B] 2022-11-29T18:01:40.632 INFO:teuthology.orchestra.run.smithi038.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.2.10-1291-g8dc57b56-1focal [49.5 MB] 2022-11-29T18:01:41.102 INFO:teuthology.orchestra.run.smithi035.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.2.10-1291-g8dc57b56-1focal [4,096 B] 2022-11-29T18:01:41.103 INFO:teuthology.orchestra.run.smithi035.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.2.10-1291-g8dc57b56-1focal [49.5 MB] 2022-11-29T18:01:44.128 INFO:teuthology.orchestra.run.smithi038.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.2.10-1291-g8dc57b56-1focal [21.4 kB] 2022-11-29T18:01:44.128 INFO:teuthology.orchestra.run.smithi038.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.2.10-1291-g8dc57b56-1focal [11.2 MB] 2022-11-29T18:01:44.730 INFO:teuthology.orchestra.run.smithi038.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.2.10-1291-g8dc57b56-1focal [78.6 kB] 2022-11-29T18:01:44.763 INFO:teuthology.orchestra.run.smithi035.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.2.10-1291-g8dc57b56-1focal [21.4 kB] 2022-11-29T18:01:44.764 INFO:teuthology.orchestra.run.smithi035.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.2.10-1291-g8dc57b56-1focal [11.2 MB] 2022-11-29T18:01:45.183 INFO:teuthology.orchestra.run.smithi035.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri3-testing-2022-11-28-0750-pacific/8dc57b5624d909c1714a82ba84ff24e49431ea76/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.2.10-1291-g8dc57b56-1focal [78.6 kB] 2022-11-29T18:01:45.345 INFO:teuthology.orchestra.run.smithi038.stdout:Fetched 174 MB in 11s (16.0 MB/s) 2022-11-29T18:01:45.453 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libjaeger. 2022-11-29T18:01:45.508 INFO:teuthology.orchestra.run.smithi038.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138843 files and directories currently installed.) 2022-11-29T18:01:45.513 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../000-libjaeger_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:45.560 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libjaeger (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:45.794 INFO:teuthology.orchestra.run.smithi035.stdout:Fetched 174 MB in 11s (15.4 MB/s) 2022-11-29T18:01:45.846 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-11-29T18:01:45.864 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../001-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-11-29T18:01:45.894 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libjaeger. 2022-11-29T18:01:45.947 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-11-29T18:01:45.949 INFO:teuthology.orchestra.run.smithi035.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138843 files and directories currently installed.) 2022-11-29T18:01:45.953 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../000-libjaeger_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:45.993 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libjaeger (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:46.317 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../002-librbd1_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:46.458 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librbd1 (16.2.10-1291-g8dc57b56-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2022-11-29T18:01:46.471 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2022-11-29T18:01:46.487 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../001-liblttng-ust0_2.11.0-1_amd64.deb ... 2022-11-29T18:01:46.571 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2022-11-29T18:01:46.950 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../002-librbd1_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:47.075 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking librbd1 (16.2.10-1291-g8dc57b56-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2022-11-29T18:01:47.153 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../003-librados2_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:47.302 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librados2 (16.2.10-1291-g8dc57b56-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2022-11-29T18:01:47.739 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../003-librados2_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:47.896 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking librados2 (16.2.10-1291-g8dc57b56-1focal) over (15.2.17-0ubuntu0.20.04.1) ... 2022-11-29T18:01:48.017 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libcephfs2. 2022-11-29T18:01:48.033 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../004-libcephfs2_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:48.075 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libcephfs2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:48.377 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-rados. 2022-11-29T18:01:48.394 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../005-python3-rados_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:48.427 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-rados (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:48.542 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libcephfs2. 2022-11-29T18:01:48.559 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../004-libcephfs2_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:48.592 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libcephfs2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:48.847 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-11-29T18:01:48.867 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../006-python3-ceph-argparse_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:01:48.903 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-rados. 2022-11-29T18:01:48.922 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../005-python3-rados_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:48.961 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-rados (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:49.055 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-ceph-argparse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:49.347 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-ceph-argparse. 2022-11-29T18:01:49.364 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../006-python3-ceph-argparse_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:01:49.374 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cephfs. 2022-11-29T18:01:49.393 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../007-python3-cephfs_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:49.406 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-ceph-argparse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:49.432 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cephfs (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:49.718 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-ceph-common. 2022-11-29T18:01:49.733 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-cephfs. 2022-11-29T18:01:49.736 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../008-python3-ceph-common_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:01:49.750 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../007-python3-cephfs_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:49.777 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-ceph-common (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:49.792 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-cephfs (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:50.086 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-ceph-common. 2022-11-29T18:01:50.105 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../008-python3-ceph-common_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:01:50.114 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-prettytable. 2022-11-29T18:01:50.130 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../009-python3-prettytable_0.7.2-5_all.deb ... 2022-11-29T18:01:50.144 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-ceph-common (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:50.205 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-11-29T18:01:50.458 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-rbd. 2022-11-29T18:01:50.475 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../010-python3-rbd_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:50.490 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-prettytable. 2022-11-29T18:01:50.507 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../009-python3-prettytable_0.7.2-5_all.deb ... 2022-11-29T18:01:50.541 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-rbd (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:50.548 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2022-11-29T18:01:50.784 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-rbd. 2022-11-29T18:01:50.800 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../010-python3-rbd_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:50.842 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-rbd (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:50.910 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2022-11-29T18:01:50.928 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../011-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2022-11-29T18:01:50.968 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2022-11-29T18:01:51.187 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2022-11-29T18:01:51.203 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../011-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2022-11-29T18:01:51.245 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2022-11-29T18:01:51.271 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-11-29T18:01:51.291 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../012-librabbitmq4_0.10.0-1_amd64.deb ... 2022-11-29T18:01:51.329 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-11-29T18:01:51.740 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package librabbitmq4:amd64. 2022-11-29T18:01:51.741 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-11-29T18:01:51.756 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../012-librabbitmq4_0.10.0-1_amd64.deb ... 2022-11-29T18:01:51.758 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../013-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-11-29T18:01:51.798 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2022-11-29T18:01:51.816 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-11-29T18:01:52.134 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package librdkafka1:amd64. 2022-11-29T18:01:52.151 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../013-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2022-11-29T18:01:52.168 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package librgw2. 2022-11-29T18:01:52.185 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../014-librgw2_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:52.215 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-11-29T18:01:52.227 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking librgw2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:52.520 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package librgw2. 2022-11-29T18:01:52.538 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../014-librgw2_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:52.579 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking librgw2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:52.887 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-rgw. 2022-11-29T18:01:52.905 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../015-python3-rgw_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:52.946 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-rgw (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:53.182 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-rgw. 2022-11-29T18:01:53.199 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../015-python3-rgw_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:53.240 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-rgw (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:53.449 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-11-29T18:01:53.465 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../016-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-11-29T18:01:53.507 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-11-29T18:01:53.592 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libleveldb1d:amd64. 2022-11-29T18:01:53.612 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../016-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2022-11-29T18:01:53.651 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-11-29T18:01:53.837 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liboath0. 2022-11-29T18:01:53.873 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../017-liboath0_2.6.1-1.3_amd64.deb ... 2022-11-29T18:01:53.885 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-11-29T18:01:53.979 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package liboath0. 2022-11-29T18:01:53.998 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../017-liboath0_2.6.1-1.3_amd64.deb ... 2022-11-29T18:01:54.062 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2022-11-29T18:01:54.204 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libradosstriper1. 2022-11-29T18:01:54.223 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../018-libradosstriper1_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:54.262 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libradosstriper1 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:54.348 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libradosstriper1. 2022-11-29T18:01:54.366 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../018-libradosstriper1_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:54.407 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libradosstriper1 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:54.749 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-common. 2022-11-29T18:01:54.768 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../019-ceph-common_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:54.807 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-common (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:54.885 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-common. 2022-11-29T18:01:54.903 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../019-ceph-common_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:54.944 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-common (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:57.046 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-base. 2022-11-29T18:01:57.064 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../020-ceph-base_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:57.106 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-base (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:57.116 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-base. 2022-11-29T18:01:57.135 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../020-ceph-base_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:01:57.176 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-base (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:58.117 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cryptography. 2022-11-29T18:01:58.136 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../021-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-11-29T18:01:58.171 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-cryptography. 2022-11-29T18:01:58.175 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-11-29T18:01:58.187 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../021-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2022-11-29T18:01:58.229 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2022-11-29T18:01:58.612 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-openssl. 2022-11-29T18:01:58.632 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../022-python3-openssl_19.0.0-1build1_all.deb ... 2022-11-29T18:01:58.670 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-11-29T18:01:58.683 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-openssl. 2022-11-29T18:01:58.699 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../022-python3-openssl_19.0.0-1build1_all.deb ... 2022-11-29T18:01:58.741 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2022-11-29T18:01:58.940 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-11-29T18:01:58.957 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../023-ceph-mgr-modules-core_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:01:58.990 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-modules-core (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:59.027 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2022-11-29T18:01:59.044 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../023-ceph-mgr-modules-core_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:01:59.086 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-mgr-modules-core (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:01:59.403 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-bcrypt. 2022-11-29T18:01:59.419 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../024-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-11-29T18:01:59.461 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-11-29T18:01:59.474 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-bcrypt. 2022-11-29T18:01:59.493 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../024-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2022-11-29T18:01:59.557 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2022-11-29T18:01:59.822 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cherrypy3. 2022-11-29T18:01:59.843 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2022-11-29T18:01:59.877 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-cherrypy3. 2022-11-29T18:01:59.881 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-11-29T18:01:59.895 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2022-11-29T18:01:59.946 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2022-11-29T18:02:00.251 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-jwt. 2022-11-29T18:02:00.268 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../026-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-11-29T18:02:00.309 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-11-29T18:02:00.351 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-jwt. 2022-11-29T18:02:00.363 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../026-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2022-11-29T18:02:00.406 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2022-11-29T18:02:00.654 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-logutils. 2022-11-29T18:02:00.670 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2022-11-29T18:02:00.712 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-11-29T18:02:00.751 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-logutils. 2022-11-29T18:02:00.770 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2022-11-29T18:02:00.809 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-logutils (0.3.3-7) ... 2022-11-29T18:02:01.040 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-markupsafe. 2022-11-29T18:02:01.060 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-11-29T18:02:01.112 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-markupsafe. 2022-11-29T18:02:01.128 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2022-11-29T18:02:01.131 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-11-29T18:02:01.175 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2022-11-29T18:02:01.478 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-mako. 2022-11-29T18:02:01.494 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2022-11-29T18:02:01.515 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-mako. 2022-11-29T18:02:01.534 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2022-11-29T18:02:01.535 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2022-11-29T18:02:01.581 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2022-11-29T18:02:01.961 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-simplegeneric. 2022-11-29T18:02:01.978 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2022-11-29T18:02:02.018 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-11-29T18:02:02.254 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-simplegeneric. 2022-11-29T18:02:02.271 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2022-11-29T18:02:02.313 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2022-11-29T18:02:02.321 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-singledispatch. 2022-11-29T18:02:02.338 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-11-29T18:02:02.379 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-11-29T18:02:02.624 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-singledispatch. 2022-11-29T18:02:02.640 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2022-11-29T18:02:02.682 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2022-11-29T18:02:02.715 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-webob. 2022-11-29T18:02:02.731 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2_all.deb ... 2022-11-29T18:02:02.774 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-11-29T18:02:03.018 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-webob. 2022-11-29T18:02:03.034 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2_all.deb ... 2022-11-29T18:02:03.092 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2022-11-29T18:02:03.102 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-soupsieve. 2022-11-29T18:02:03.121 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-11-29T18:02:03.185 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-11-29T18:02:03.431 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-soupsieve. 2022-11-29T18:02:03.448 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2022-11-29T18:02:03.487 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2022-11-29T18:02:03.514 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-bs4. 2022-11-29T18:02:03.530 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2022-11-29T18:02:03.571 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-11-29T18:02:03.823 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-bs4. 2022-11-29T18:02:03.843 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2022-11-29T18:02:03.881 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2022-11-29T18:02:03.950 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-waitress. 2022-11-29T18:02:03.975 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-11-29T18:02:04.011 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-11-29T18:02:04.259 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-waitress. 2022-11-29T18:02:04.277 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2022-11-29T18:02:04.354 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2022-11-29T18:02:04.370 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-tempita. 2022-11-29T18:02:04.389 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2022-11-29T18:02:04.436 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-11-29T18:02:04.721 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-tempita. 2022-11-29T18:02:04.737 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2022-11-29T18:02:04.739 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-paste. 2022-11-29T18:02:04.755 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-11-29T18:02:04.779 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-tempita (0.5.2-6) ... 2022-11-29T18:02:04.797 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-11-29T18:02:05.116 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-paste. 2022-11-29T18:02:05.133 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2022-11-29T18:02:05.159 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-11-29T18:02:05.173 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2022-11-29T18:02:05.175 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-11-29T18:02:05.217 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-11-29T18:02:05.493 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2022-11-29T18:02:05.511 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2022-11-29T18:02:05.520 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-pastedeploy. 2022-11-29T18:02:05.536 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2022-11-29T18:02:05.551 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2022-11-29T18:02:05.611 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-11-29T18:02:05.896 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pastedeploy. 2022-11-29T18:02:05.913 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2022-11-29T18:02:05.924 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-webtest. 2022-11-29T18:02:05.939 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2022-11-29T18:02:05.971 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2022-11-29T18:02:05.982 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-11-29T18:02:06.316 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-webtest. 2022-11-29T18:02:06.326 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-pecan. 2022-11-29T18:02:06.333 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2022-11-29T18:02:06.343 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-11-29T18:02:06.377 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-webtest (2.0.32-2) ... 2022-11-29T18:02:06.393 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-11-29T18:02:06.710 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pecan. 2022-11-29T18:02:06.728 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2022-11-29T18:02:06.765 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-werkzeug. 2022-11-29T18:02:06.769 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2022-11-29T18:02:06.785 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-11-29T18:02:07.373 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-werkzeug. 2022-11-29T18:02:07.392 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2022-11-29T18:02:07.400 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-11-29T18:02:07.843 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2022-11-29T18:02:07.856 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2022-11-29T18:02:07.859 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../043-libsqlite3-mod-ceph_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:07.902 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libsqlite3-mod-ceph (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:08.251 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2022-11-29T18:02:08.269 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../043-libsqlite3-mod-ceph_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:08.296 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-mgr. 2022-11-29T18:02:08.314 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../044-ceph-mgr_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:08.343 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libsqlite3-mod-ceph (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:08.354 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-mgr (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:08.721 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr. 2022-11-29T18:02:08.737 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../044-ceph-mgr_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:08.779 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:08.858 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-mon. 2022-11-29T18:02:08.875 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../045-ceph-mon_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:08.916 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-mon (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:09.343 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mon. 2022-11-29T18:02:09.363 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../045-ceph-mon_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:09.395 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mon (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:09.986 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-osd. 2022-11-29T18:02:10.007 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../046-ceph-osd_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:10.044 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-osd (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:10.655 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-osd. 2022-11-29T18:02:10.672 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../046-ceph-osd_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:10.712 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-osd (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:12.767 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph. 2022-11-29T18:02:12.783 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../047-ceph_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:12.825 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:13.144 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-fuse. 2022-11-29T18:02:13.163 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../048-ceph-fuse_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:13.204 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-fuse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:13.529 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph. 2022-11-29T18:02:13.548 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../047-ceph_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:13.592 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:13.598 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-mds. 2022-11-29T18:02:13.617 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../049-ceph-mds_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:13.672 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-mds (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:13.945 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-fuse. 2022-11-29T18:02:13.961 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../048-ceph-fuse_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:14.003 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-fuse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:14.193 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package cephadm. 2022-11-29T18:02:14.208 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../050-cephadm_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:14.276 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking cephadm (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:14.448 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mds. 2022-11-29T18:02:14.467 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../049-ceph-mds_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:14.540 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mds (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:14.604 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-jinja2. 2022-11-29T18:02:14.620 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-11-29T18:02:14.662 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-11-29T18:02:15.024 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-11-29T18:02:15.040 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:02:15.060 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package cephadm. 2022-11-29T18:02:15.079 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../050-cephadm_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:15.082 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-mgr-cephadm (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:15.109 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking cephadm (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:15.457 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-jinja2. 2022-11-29T18:02:15.469 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-repoze.lru. 2022-11-29T18:02:15.475 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2022-11-29T18:02:15.488 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-11-29T18:02:15.520 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2022-11-29T18:02:15.535 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-11-29T18:02:15.805 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libjs-underscore. 2022-11-29T18:02:15.825 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-11-29T18:02:15.857 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2022-11-29T18:02:15.863 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-11-29T18:02:15.876 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:02:15.915 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-cephadm (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:16.151 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-11-29T18:02:16.169 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-11-29T18:02:16.209 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-11-29T18:02:16.293 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-repoze.lru. 2022-11-29T18:02:16.309 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2022-11-29T18:02:16.351 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2022-11-29T18:02:16.537 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-routes. 2022-11-29T18:02:16.555 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-11-29T18:02:16.606 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-11-29T18:02:16.629 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libjs-underscore. 2022-11-29T18:02:16.650 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2022-11-29T18:02:16.687 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-11-29T18:02:16.974 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-11-29T18:02:16.981 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libjs-sphinxdoc. 2022-11-29T18:02:16.990 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:02:16.998 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2022-11-29T18:02:17.032 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-mgr-dashboard (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:17.065 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-11-29T18:02:17.410 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-routes. 2022-11-29T18:02:17.426 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2022-11-29T18:02:17.468 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-routes (2.4.1-2) ... 2022-11-29T18:02:17.813 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2022-11-29T18:02:17.829 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:02:17.871 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-dashboard (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:18.582 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-decorator. 2022-11-29T18:02:18.599 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../058-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-11-29T18:02:18.639 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-11-29T18:02:18.959 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-11-29T18:02:18.976 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../059-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-11-29T18:02:19.012 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-decorator. 2022-11-29T18:02:19.017 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-11-29T18:02:19.030 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../058-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2022-11-29T18:02:19.070 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2022-11-29T18:02:19.362 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-scipy. 2022-11-29T18:02:19.380 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../060-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-11-29T18:02:19.420 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-11-29T18:02:19.432 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2022-11-29T18:02:19.449 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../059-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2022-11-29T18:02:19.498 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-11-29T18:02:19.835 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-scipy. 2022-11-29T18:02:19.851 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../060-python3-scipy_1.3.3-3build1_amd64.deb ... 2022-11-29T18:02:19.893 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2022-11-29T18:02:20.871 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-11-29T18:02:20.888 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-11-29T18:02:20.946 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-11-29T18:02:21.403 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-sklearn-lib. 2022-11-29T18:02:21.419 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2022-11-29T18:02:21.461 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-11-29T18:02:21.491 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-joblib. 2022-11-29T18:02:21.508 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../062-python3-joblib_0.14.0-3_all.deb ... 2022-11-29T18:02:21.583 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-11-29T18:02:21.970 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-sklearn. 2022-11-29T18:02:21.987 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../063-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-11-29T18:02:22.028 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-11-29T18:02:22.065 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-joblib. 2022-11-29T18:02:22.084 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../062-python3-joblib_0.14.0-3_all.deb ... 2022-11-29T18:02:22.126 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-joblib (0.14.0-3) ... 2022-11-29T18:02:22.543 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-sklearn. 2022-11-29T18:02:22.561 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../063-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2022-11-29T18:02:22.601 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-11-29T18:02:22.720 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-11-29T18:02:22.736 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:02:22.803 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:23.410 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2022-11-29T18:02:23.426 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:02:23.468 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:24.090 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-cachetools. 2022-11-29T18:02:24.107 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../065-python3-cachetools_4.0.0-1_all.deb ... 2022-11-29T18:02:24.156 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-11-29T18:02:24.484 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-pyasn1. 2022-11-29T18:02:24.501 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../066-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-11-29T18:02:24.542 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-11-29T18:02:24.671 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-cachetools. 2022-11-29T18:02:24.688 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../065-python3-cachetools_4.0.0-1_all.deb ... 2022-11-29T18:02:24.737 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2022-11-29T18:02:24.897 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-11-29T18:02:24.914 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../067-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-11-29T18:02:24.955 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-11-29T18:02:25.048 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pyasn1. 2022-11-29T18:02:25.067 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../066-python3-pyasn1_0.4.2-3build1_all.deb ... 2022-11-29T18:02:25.107 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2022-11-29T18:02:25.308 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-rsa. 2022-11-29T18:02:25.328 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../068-python3-rsa_4.0-3_all.deb ... 2022-11-29T18:02:25.366 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-rsa (4.0-3) ... 2022-11-29T18:02:25.460 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pyasn1-modules. 2022-11-29T18:02:25.477 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../067-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2022-11-29T18:02:25.518 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-11-29T18:02:25.668 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-google-auth. 2022-11-29T18:02:25.681 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../069-python3-google-auth_1.5.1-2_all.deb ... 2022-11-29T18:02:25.719 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-11-29T18:02:25.871 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-rsa. 2022-11-29T18:02:25.888 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../068-python3-rsa_4.0-3_all.deb ... 2022-11-29T18:02:25.929 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-rsa (4.0-3) ... 2022-11-29T18:02:26.064 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-blinker. 2022-11-29T18:02:26.080 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../070-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-11-29T18:02:26.122 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-11-29T18:02:26.276 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-google-auth. 2022-11-29T18:02:26.293 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../069-python3-google-auth_1.5.1-2_all.deb ... 2022-11-29T18:02:26.349 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2022-11-29T18:02:26.425 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-oauthlib. 2022-11-29T18:02:26.442 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../071-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-11-29T18:02:26.500 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-11-29T18:02:26.728 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-blinker. 2022-11-29T18:02:26.745 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../070-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2022-11-29T18:02:26.786 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-11-29T18:02:26.837 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-11-29T18:02:26.855 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../072-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-11-29T18:02:26.895 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-11-29T18:02:27.114 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-oauthlib. 2022-11-29T18:02:27.130 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../071-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2022-11-29T18:02:27.172 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2022-11-29T18:02:27.231 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-websocket. 2022-11-29T18:02:27.248 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../073-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-11-29T18:02:27.289 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-11-29T18:02:27.551 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-requests-oauthlib. 2022-11-29T18:02:27.567 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../072-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2022-11-29T18:02:27.609 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-kubernetes. 2022-11-29T18:02:27.627 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../074-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-11-29T18:02:27.628 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-11-29T18:02:27.693 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-11-29T18:02:27.970 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-websocket. 2022-11-29T18:02:27.988 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../073-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2022-11-29T18:02:28.028 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2022-11-29T18:02:28.256 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-11-29T18:02:28.274 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../075-ceph-mgr-k8sevents_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:02:28.322 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-mgr-k8sevents (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:28.365 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-kubernetes. 2022-11-29T18:02:28.382 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../074-python3-kubernetes_7.0.0~a1-2_all.deb ... 2022-11-29T18:02:28.423 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2022-11-29T18:02:28.641 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libonig5:amd64. 2022-11-29T18:02:28.659 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../076-libonig5_6.9.4-1_amd64.deb ... 2022-11-29T18:02:28.708 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-11-29T18:02:29.003 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2022-11-29T18:02:29.021 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../075-ceph-mgr-k8sevents_16.2.10-1291-g8dc57b56-1focal_all.deb ... 2022-11-29T18:02:29.061 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-mgr-k8sevents (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:29.086 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libjq1:amd64. 2022-11-29T18:02:29.105 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../077-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-11-29T18:02:29.144 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-11-29T18:02:29.430 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libonig5:amd64. 2022-11-29T18:02:29.439 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package jq. 2022-11-29T18:02:29.447 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../076-libonig5_6.9.4-1_amd64.deb ... 2022-11-29T18:02:29.457 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../078-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-11-29T18:02:29.509 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2022-11-29T18:02:29.530 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-11-29T18:02:29.783 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package socat. 2022-11-29T18:02:29.803 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../079-socat_1.7.3.3-2_amd64.deb ... 2022-11-29T18:02:29.841 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking socat (1.7.3.3-2) ... 2022-11-29T18:02:29.900 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libjq1:amd64. 2022-11-29T18:02:29.917 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../077-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-11-29T18:02:29.958 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-11-29T18:02:30.261 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package jq. 2022-11-29T18:02:30.263 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-11-29T18:02:30.279 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../080-libxslt1.1_1.1.34-4ubuntu0.20.04.1_amd64.deb ... 2022-11-29T18:02:30.279 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../078-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2022-11-29T18:02:30.321 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4ubuntu0.20.04.1) ... 2022-11-29T18:02:30.324 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2022-11-29T18:02:30.605 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package socat. 2022-11-29T18:02:30.624 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../079-socat_1.7.3.3-2_amd64.deb ... 2022-11-29T18:02:30.677 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking socat (1.7.3.3-2) ... 2022-11-29T18:02:30.740 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package xmlstarlet. 2022-11-29T18:02:30.757 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../081-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-11-29T18:02:30.799 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-11-29T18:02:31.134 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libxslt1.1:amd64. 2022-11-29T18:02:31.150 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../080-libxslt1.1_1.1.34-4ubuntu0.20.04.1_amd64.deb ... 2022-11-29T18:02:31.153 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package ceph-test. 2022-11-29T18:02:31.171 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../082-ceph-test_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:31.200 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4ubuntu0.20.04.1) ... 2022-11-29T18:02:31.219 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking ceph-test (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:31.580 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package xmlstarlet. 2022-11-29T18:02:31.597 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../081-xmlstarlet_1.6.1-2build1_amd64.deb ... 2022-11-29T18:02:31.638 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2022-11-29T18:02:32.000 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package ceph-test. 2022-11-29T18:02:32.017 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../082-ceph-test_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:32.092 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking ceph-test (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:35.531 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package libcephfs-dev. 2022-11-29T18:02:35.550 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../083-libcephfs-dev_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:35.581 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking libcephfs-dev (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:36.001 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package nvme-cli. 2022-11-29T18:02:36.019 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../084-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-11-29T18:02:36.077 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-11-29T18:02:36.481 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-atomicwrites. 2022-11-29T18:02:36.498 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../085-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-11-29T18:02:36.540 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-11-29T18:02:36.554 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package libcephfs-dev. 2022-11-29T18:02:36.574 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../083-libcephfs-dev_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:36.612 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking libcephfs-dev (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:36.858 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-attr. 2022-11-29T18:02:36.877 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../086-python3-attr_19.3.0-2_all.deb ... 2022-11-29T18:02:36.916 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-11-29T18:02:37.048 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package nvme-cli. 2022-11-29T18:02:37.065 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../084-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2022-11-29T18:02:37.124 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2022-11-29T18:02:37.236 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-webencodings. 2022-11-29T18:02:37.256 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../087-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-11-29T18:02:37.294 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-11-29T18:02:37.462 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-atomicwrites. 2022-11-29T18:02:37.479 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../085-python3-atomicwrites_1.1.5-2build1_all.deb ... 2022-11-29T18:02:37.520 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2022-11-29T18:02:37.622 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-html5lib. 2022-11-29T18:02:37.639 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../088-python3-html5lib_1.0.1-2_all.deb ... 2022-11-29T18:02:37.680 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-11-29T18:02:37.832 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-attr. 2022-11-29T18:02:37.851 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../086-python3-attr_19.3.0-2_all.deb ... 2022-11-29T18:02:37.889 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-attr (19.3.0-2) ... 2022-11-29T18:02:38.058 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-11-29T18:02:38.077 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../089-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-11-29T18:02:38.116 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-11-29T18:02:38.259 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-webencodings. 2022-11-29T18:02:38.276 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../087-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2022-11-29T18:02:38.342 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2022-11-29T18:02:38.571 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-packaging. 2022-11-29T18:02:38.587 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../090-python3-packaging_20.3-1_all.deb ... 2022-11-29T18:02:38.645 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-html5lib. 2022-11-29T18:02:38.648 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-packaging (20.3-1) ... 2022-11-29T18:02:38.664 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../088-python3-html5lib_1.0.1-2_all.deb ... 2022-11-29T18:02:38.695 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2022-11-29T18:02:38.931 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-pastescript. 2022-11-29T18:02:38.950 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../091-python3-pastescript_2.0.2-4_all.deb ... 2022-11-29T18:02:38.989 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-11-29T18:02:39.031 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-lxml:amd64. 2022-11-29T18:02:39.048 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../089-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2022-11-29T18:02:39.089 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-11-29T18:02:39.309 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-pluggy. 2022-11-29T18:02:39.327 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../092-python3-pluggy_0.13.0-2_all.deb ... 2022-11-29T18:02:39.367 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-11-29T18:02:39.543 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-packaging. 2022-11-29T18:02:39.561 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../090-python3-packaging_20.3-1_all.deb ... 2022-11-29T18:02:39.593 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-packaging (20.3-1) ... 2022-11-29T18:02:39.670 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-psutil. 2022-11-29T18:02:39.687 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../093-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-11-29T18:02:39.728 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-11-29T18:02:39.895 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pastescript. 2022-11-29T18:02:39.912 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../091-python3-pastescript_2.0.2-4_all.deb ... 2022-11-29T18:02:39.979 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2022-11-29T18:02:40.090 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-py. 2022-11-29T18:02:40.107 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../094-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-11-29T18:02:40.148 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-11-29T18:02:40.343 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pluggy. 2022-11-29T18:02:40.361 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../092-python3-pluggy_0.13.0-2_all.deb ... 2022-11-29T18:02:40.399 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2022-11-29T18:02:40.518 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-pyinotify. 2022-11-29T18:02:40.538 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../095-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-11-29T18:02:40.576 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-11-29T18:02:40.727 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-psutil. 2022-11-29T18:02:40.744 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../093-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2022-11-29T18:02:40.785 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2022-11-29T18:02:40.904 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-wcwidth. 2022-11-29T18:02:40.921 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../096-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-11-29T18:02:40.962 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-11-29T18:02:41.147 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-py. 2022-11-29T18:02:41.164 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../094-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2022-11-29T18:02:41.230 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2022-11-29T18:02:41.290 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-pytest. 2022-11-29T18:02:41.307 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../097-python3-pytest_4.6.9-1_all.deb ... 2022-11-29T18:02:41.348 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-11-29T18:02:41.575 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pyinotify. 2022-11-29T18:02:41.593 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../095-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2022-11-29T18:02:41.633 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-11-29T18:02:41.711 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package python3-simplejson. 2022-11-29T18:02:41.729 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../098-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-11-29T18:02:41.768 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-11-29T18:02:41.970 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-wcwidth. 2022-11-29T18:02:41.986 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../096-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2022-11-29T18:02:42.027 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2022-11-29T18:02:42.205 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package radosgw. 2022-11-29T18:02:42.223 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../099-radosgw_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:42.263 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking radosgw (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:42.330 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-pytest. 2022-11-29T18:02:42.351 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../097-python3-pytest_4.6.9-1_all.deb ... 2022-11-29T18:02:42.389 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-pytest (4.6.9-1) ... 2022-11-29T18:02:42.785 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package python3-simplejson. 2022-11-29T18:02:42.804 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../098-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2022-11-29T18:02:42.859 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2022-11-29T18:02:43.354 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package radosgw. 2022-11-29T18:02:43.374 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../099-radosgw_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:43.412 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking radosgw (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:43.575 INFO:teuthology.orchestra.run.smithi035.stdout:Selecting previously unselected package rbd-fuse. 2022-11-29T18:02:43.591 INFO:teuthology.orchestra.run.smithi035.stdout:Preparing to unpack .../100-rbd-fuse_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:43.641 INFO:teuthology.orchestra.run.smithi035.stdout:Unpacking rbd-fuse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:43.942 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-attr (19.3.0-2) ... 2022-11-29T18:02:44.271 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-11-29T18:02:44.397 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-11-29T18:02:44.523 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-11-29T18:02:44.674 INFO:teuthology.orchestra.run.smithi038.stdout:Selecting previously unselected package rbd-fuse. 2022-11-29T18:02:44.694 INFO:teuthology.orchestra.run.smithi038.stdout:Preparing to unpack .../100-rbd-fuse_16.2.10-1291-g8dc57b56-1focal_amd64.deb ... 2022-11-29T18:02:44.732 INFO:teuthology.orchestra.run.smithi038.stdout:Unpacking rbd-fuse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:44.767 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up cephadm (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:45.057 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-attr (19.3.0-2) ... 2022-11-29T18:02:45.385 INFO:teuthology.orchestra.run.smithi035.stdout:Adding system user cephadm....done 2022-11-29T18:02:45.437 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2022-11-29T18:02:45.557 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-11-29T18:02:45.580 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2022-11-29T18:02:45.724 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2022-11-29T18:02:45.916 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-11-29T18:02:45.923 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up cephadm (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:46.242 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-11-29T18:02:46.393 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-11-29T18:02:46.568 INFO:teuthology.orchestra.run.smithi038.stdout:Adding system user cephadm....done 2022-11-29T18:02:46.719 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-11-29T18:02:46.755 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2022-11-29T18:02:47.114 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-repoze.lru (0.7-2) ... 2022-11-29T18:02:47.129 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-11-29T18:02:47.407 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liboath0 (2.6.1-1.3) ... 2022-11-29T18:02:47.456 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-11-29T18:02:47.517 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2022-11-29T18:02:47.757 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-11-29T18:02:47.852 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-joblib (0.14.0-3) ... 2022-11-29T18:02:48.092 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-11-29T18:02:48.245 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2022-11-29T18:02:48.418 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-ceph-argparse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:48.612 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cachetools (4.0.0-1) ... 2022-11-29T18:02:48.948 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2022-11-29T18:02:49.031 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-11-29T18:02:49.153 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-11-29T18:02:49.282 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2022-11-29T18:02:49.429 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-11-29T18:02:49.591 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-ceph-argparse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:49.722 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-11-29T18:02:49.918 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2022-11-29T18:02:50.052 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2022-11-29T18:02:50.098 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-11-29T18:02:50.337 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2022-11-29T18:02:50.400 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-11-29T18:02:50.638 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2022-11-29T18:02:50.760 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-11-29T18:02:51.014 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2022-11-29T18:02:51.086 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libjaeger (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:51.212 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-11-29T18:02:51.290 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2022-11-29T18:02:51.554 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-11-29T18:02:51.642 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2022-11-29T18:02:51.839 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-packaging (20.3-1) ... 2022-11-29T18:02:51.935 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libjaeger (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:52.061 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2022-11-29T18:02:52.124 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-11-29T18:02:52.283 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-11-29T18:02:52.453 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-tempita (0.5.2-6) ... 2022-11-29T18:02:52.576 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-11-29T18:02:52.797 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-packaging (20.3-1) ... 2022-11-29T18:02:53.003 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-11-29T18:02:53.139 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2022-11-29T18:02:53.265 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2022-11-29T18:02:53.387 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-11-29T18:02:53.558 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2022-11-29T18:02:53.773 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up socat (1.7.3.3-2) ... 2022-11-29T18:02:53.923 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-ceph-common (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:53.960 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2022-11-29T18:02:54.259 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-11-29T18:02:54.478 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2022-11-29T18:02:54.593 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-11-29T18:02:54.719 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-11-29T18:02:54.855 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up socat (1.7.3.3-2) ... 2022-11-29T18:02:54.981 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-ceph-common (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:55.020 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2022-11-29T18:02:55.180 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libxslt1.1:amd64 (1.1.34-4ubuntu0.20.04.1) ... 2022-11-29T18:02:55.306 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-11-29T18:02:55.333 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-html5lib (1.0.1-2) ... 2022-11-29T18:02:55.649 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-11-29T18:02:55.737 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2022-11-29T18:02:55.862 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pluggy (0.13.0-2) ... 2022-11-29T18:02:55.959 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-11-29T18:02:56.155 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2022-11-29T18:02:56.317 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-11-29T18:02:56.318 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libxslt1.1:amd64 (1.1.34-4ubuntu0.20.04.1) ... 2022-11-29T18:02:56.434 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2022-11-29T18:02:56.436 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-11-29T18:02:56.703 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-11-29T18:02:56.759 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2022-11-29T18:02:57.005 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-11-29T18:02:57.052 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-logutils (0.3.3-7) ... 2022-11-29T18:02:57.298 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-11-29T18:02:57.362 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2022-11-29T18:02:57.488 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2022-11-29T18:02:57.592 INFO:teuthology.orchestra.run.smithi035.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-11-29T18:02:57.781 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-prettytable (0.7.2-5) ... 2022-11-29T18:02:57.875 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-11-29T18:02:58.034 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2022-11-29T18:02:58.081 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2022-11-29T18:02:58.361 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-11-29T18:02:58.407 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2022-11-29T18:02:58.701 INFO:teuthology.orchestra.run.smithi038.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2022-11-29T18:02:58.720 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-11-29T18:02:58.863 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-11-29T18:02:58.994 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2022-11-29T18:02:58.997 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-11-29T18:02:59.120 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2022-11-29T18:02:59.324 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-11-29T18:02:59.438 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2022-11-29T18:02:59.641 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up librados2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:02:59.784 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-11-29T18:02:59.822 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2022-11-29T18:02:59.982 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2022-11-29T18:03:00.108 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-webob (1:1.8.5-2) ... 2022-11-29T18:03:00.476 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2022-11-29T18:03:00.794 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librados2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:00.964 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2022-11-29T18:03:01.290 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up librgw2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:01.416 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libsqlite3-mod-ceph (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:01.550 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-11-29T18:03:01.701 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-11-29T18:03:02.108 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-11-29T18:03:02.438 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-11-29T18:03:02.469 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librgw2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:02.594 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libsqlite3-mod-ceph (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:02.728 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2022-11-29T18:03:02.730 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-11-29T18:03:02.879 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pytest (4.6.9-1) ... 2022-11-29T18:03:03.082 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libcephfs2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:03.241 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-11-29T18:03:03.256 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2022-11-29T18:03:03.557 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2022-11-29T18:03:03.568 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libradosstriper1 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:03.719 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-11-29T18:03:03.850 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-bs4 (4.8.2-1) ... 2022-11-29T18:03:04.054 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-11-29T18:03:04.202 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libcephfs2 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:04.328 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2022-11-29T18:03:04.722 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libradosstriper1 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:04.864 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2022-11-29T18:03:05.142 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up librbd1 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:05.199 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2022-11-29T18:03:05.278 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-11-29T18:03:05.405 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-11-29T18:03:05.564 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-11-29T18:03:05.890 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-11-29T18:03:06.048 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-rsa (4.0-3) ... 2022-11-29T18:03:06.296 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up librbd1 (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:06.326 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-11-29T18:03:06.430 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2022-11-29T18:03:06.565 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2022-11-29T18:03:06.660 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-mgr-modules-core (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:06.707 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2022-11-29T18:03:06.795 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-fuse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:07.021 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-11-29T18:03:07.022 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-11-29T18:03:07.034 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2022-11-29T18:03:07.160 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-rsa (4.0-3) ... 2022-11-29T18:03:07.446 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up libcephfs-dev (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:07.495 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-webtest (2.0.32-2) ... 2022-11-29T18:03:07.564 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-11-29T18:03:07.838 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-modules-core (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:07.907 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-11-29T18:03:07.989 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-fuse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:08.216 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-11-29T18:03:08.228 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-11-29T18:03:08.229 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2022-11-29T18:03:08.594 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-rados (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:08.691 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up libcephfs-dev (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:08.728 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-rbd (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:08.839 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2022-11-29T18:03:08.846 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up rbd-fuse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:08.947 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-routes (2.4.1-2) ... 2022-11-29T18:03:09.159 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pastescript (2.0.2-4) ... 2022-11-29T18:03:09.366 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-11-29T18:03:09.520 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2022-11-29T18:03:09.709 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-rgw (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:09.835 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-cephfs (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:10.128 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-11-29T18:03:10.163 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-rados (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:10.298 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-rbd (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:10.424 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up rbd-fuse (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:10.583 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-routes (2.4.1-2) ... 2022-11-29T18:03:10.961 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-google-auth (1.5.1-2) ... 2022-11-29T18:03:11.073 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-common (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:11.263 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-rgw (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:11.429 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-cephfs (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:11.634 INFO:teuthology.orchestra.run.smithi035.stdout:Adding group ceph....done 2022-11-29T18:03:11.715 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2022-11-29T18:03:12.011 INFO:teuthology.orchestra.run.smithi035.stdout:Adding system user ceph....done 2022-11-29T18:03:12.212 INFO:teuthology.orchestra.run.smithi035.stdout:Setting system user ceph properties....done 2022-11-29T18:03:12.234 INFO:teuthology.orchestra.run.smithi035.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-11-29T18:03:12.389 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-11-29T18:03:12.708 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-common (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:12.720 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-11-29T18:03:13.174 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-test (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:13.300 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up radosgw (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:13.305 INFO:teuthology.orchestra.run.smithi038.stdout:Adding group ceph....done 2022-11-29T18:03:13.611 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-11-29T18:03:13.611 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-11-29T18:03:13.707 INFO:teuthology.orchestra.run.smithi038.stdout:Adding system user ceph....done 2022-11-29T18:03:13.883 INFO:teuthology.orchestra.run.smithi038.stdout:Setting system user ceph properties....done 2022-11-29T18:03:13.903 INFO:teuthology.orchestra.run.smithi038.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2022-11-29T18:03:14.055 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2022-11-29T18:03:14.269 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-base (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:14.394 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2022-11-29T18:03:14.659 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-11-29T18:03:14.879 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-test (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:15.038 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up radosgw (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:15.121 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-mds (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:15.319 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-11-29T18:03:15.319 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2022-11-29T18:03:15.368 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-11-29T18:03:15.369 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-11-29T18:03:15.823 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-mgr (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:15.975 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-base (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:16.047 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-11-29T18:03:16.048 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-11-29T18:03:16.524 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-osd (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:16.720 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2022-11-29T18:03:17.033 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-11-29T18:03:17.034 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-11-29T18:03:17.186 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mds (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:17.436 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-11-29T18:03:17.437 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2022-11-29T18:03:17.484 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-mgr-k8sevents (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:17.635 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-mgr-diskprediction-local (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:17.777 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-mon (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:17.896 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:18.003 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-11-29T18:03:18.004 INFO:teuthology.orchestra.run.smithi035.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-11-29T18:03:18.121 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-11-29T18:03:18.122 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2022-11-29T18:03:18.496 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-mgr-cephadm (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:18.589 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-osd (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:18.655 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:18.781 INFO:teuthology.orchestra.run.smithi035.stdout:Setting up ceph-mgr-dashboard (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:18.950 INFO:teuthology.orchestra.run.smithi035.stdout:Processing triggers for systemd (245.4-4ubuntu3.18) ... 2022-11-29T18:03:19.090 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-11-29T18:03:19.090 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2022-11-29T18:03:19.336 INFO:teuthology.orchestra.run.smithi035.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-11-29T18:03:19.557 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-k8sevents (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:19.666 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-diskprediction-local (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:19.843 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mon (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:20.065 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-11-29T18:03:20.066 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2022-11-29T18:03:20.528 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-cephadm (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:20.671 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:20.822 INFO:teuthology.orchestra.run.smithi038.stdout:Setting up ceph-mgr-dashboard (16.2.10-1291-g8dc57b56-1focal) ... 2022-11-29T18:03:20.965 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for systemd (245.4-4ubuntu3.18) ... 2022-11-29T18:03:21.258 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for man-db (2.9.1-1) ... 2022-11-29T18:03:22.370 INFO:teuthology.orchestra.run.smithi035.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-11-29T18:03:22.851 INFO:teuthology.orchestra.run.smithi035.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-11-29T18:03:22.855 DEBUG:teuthology.parallel:result is None 2022-11-29T18:03:24.035 INFO:teuthology.orchestra.run.smithi038.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2022-11-29T18:03:24.503 INFO:teuthology.orchestra.run.smithi038.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2022-11-29T18:03:24.505 DEBUG:teuthology.parallel:result is None 2022-11-29T18:03:24.506 DEBUG:teuthology.orchestra.run.smithi035:> sudo lsb_release -is 2022-11-29T18:03:24.560 INFO:teuthology.orchestra.run.smithi035.stdout:Ubuntu 2022-11-29T18:03:24.565 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-11-29T18:03:24.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=8dc57b5624d909c1714a82ba84ff24e49431ea76 2022-11-29T18:03:24.722 DEBUG:teuthology.orchestra.run.smithi035:> dpkg-query -W -f '${Version}' ceph 2022-11-29T18:03:24.746 INFO:teuthology.orchestra.run.smithi035.stdout:16.2.10-1291-g8dc57b56-1focal 2022-11-29T18:03:24.747 INFO:teuthology.packaging:The installed version of ceph is 16.2.10-1291-g8dc57b56-1focal 2022-11-29T18:03:24.748 INFO:teuthology.task.install:The correct ceph version 16.2.10-1291-g8dc57b56-1focal is installed. 2022-11-29T18:03:24.749 DEBUG:teuthology.orchestra.run.smithi038:> sudo lsb_release -is 2022-11-29T18:03:24.800 INFO:teuthology.orchestra.run.smithi038.stdout:Ubuntu 2022-11-29T18:03:24.806 DEBUG:teuthology.misc:System to be installed: Ubuntu 2022-11-29T18:03:24.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=8dc57b5624d909c1714a82ba84ff24e49431ea76 2022-11-29T18:03:24.961 DEBUG:teuthology.orchestra.run.smithi038:> dpkg-query -W -f '${Version}' ceph 2022-11-29T18:03:24.985 INFO:teuthology.orchestra.run.smithi038.stdout:16.2.10-1291-g8dc57b56-1focal 2022-11-29T18:03:24.986 INFO:teuthology.packaging:The installed version of ceph is 16.2.10-1291-g8dc57b56-1focal 2022-11-29T18:03:24.987 INFO:teuthology.task.install:The correct ceph version 16.2.10-1291-g8dc57b56-1focal is installed. 2022-11-29T18:03:24.988 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-29T18:03:24.989 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:24.989 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-29T18:03:25.008 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:25.009 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-29T18:03:25.043 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-29T18:03:25.043 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:25.044 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/usr/bin/daemon-helper 2022-11-29T18:03:25.061 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-29T18:03:25.123 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:25.124 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2022-11-29T18:03:25.141 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-29T18:03:25.198 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-29T18:03:25.198 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:25.199 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-29T18:03:25.216 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-29T18:03:25.273 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:25.273 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-29T18:03:25.290 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-29T18:03:25.348 INFO:teuthology.run_tasks:Running task ceph... 2022-11-29T18:03:25.426 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-29T18:03:25.427 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 777 /var/log/ceph 2022-11-29T18:03:25.430 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /var/log/ceph 2022-11-29T18:03:25.445 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-29T18:03:25.445 DEBUG:teuthology.orchestra.run.smithi035:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-29T18:03:25.492 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-29T18:03:25.508 INFO:tasks.ceph:Creating extra log directories... 2022-11-29T18:03:25.509 DEBUG:teuthology.orchestra.run.smithi035:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-29T18:03:25.512 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-29T18:03:25.532 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-11-29T18:03:25.533 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-11-29T18:03:25.533 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-11-29T18:03:25.533 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:25.534 DEBUG:teuthology.orchestra.run.smithi035:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-29T18:03:25.576 DEBUG:teuthology.orchestra.run.smithi035:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-29T18:03:25.617 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:25.617 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-29T18:03:25.625 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-29T18:03:25.669 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-29T18:03:25.670 INFO:tasks.ceph:config {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 5}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_', 'replacing it with standby', 'No standby daemons available', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-29T18:03:25.670 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi/7096909', 'branch': 'wip-yuri3-testing-2022-11-28-0750-pacific', 'description': 'rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity objectstore/bluestore-hybrid supported-random-distro$/{ubuntu_latest} tasks/failover}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7096909', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-11-29_15:35:32-rados-wip-yuri3-testing-2022-11-28-0750-pacific-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 2, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2022-11-28-0750-pacific'}, 'ceph': {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 5}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '8dc57b5624d909c1714a82ba84ff24e49431ea76'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8dc57b5624d909c1714a82ba84ff24e49431ea76'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2022-11-28-0750-pacific', 'sha1': '8dc57b5624d909c1714a82ba84ff24e49431ea76'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 61, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'], ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1']], 'sha1': '8dc57b5624d909c1714a82ba84ff24e49431ea76', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri3-testing-2022-11-28-0750-pacific', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8dc57b5624d909c1714a82ba84ff24e49431ea76/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8dc57b5624d909c1714a82ba84ff24e49431ea76', 'targets': {'smithi035.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtXAqgNSk3fbyauaAQ087MhP5pRS0EkYRcTNIno42bLKmY/5pysgUhxeVeBieflZ9GyMQdZn31ljFuM7O3aS0s/t+1nXnY4XhcvDDOO2eTUJKi0S/vviK1siPe1mR/aT8aCt38YDM7Hndl/PoU57GJc2K/RCANSTfyESt4x7TvkPTDsH6G13FV8TNL3D+hXCI2BugeRrLfkYzHwG4BaBON196sOtq80yagvFtCziW95BYvjprs6+mC0t6Csaw+uXHlhCHuh+M2nVFo4vH5U+TX53778kJ2DGe4X6yLRAeb7EZizfPO3D++zbSvroobAKjJioLD9HmIJFUDS5JZmizjQEuSkLk0MhMOY/tW6xVMXWi78qywZmWEmaMeAunbqrKaZLmF6/kZFy2iytetJTwTmw4KaTNbGCzCx5nDkicHX1N6JxN4r0LhuRRXkSYlPnRslPttny/9phAXcxO62p2i7EIC8pSFabccsgBniB5BEyEqJk2NzvCOt6ReFVpCKOM=', 'smithi038.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKIsYmsaWdEaM9muLVCDHUqCdhAlsCmeYHg9Gh/4/zAQDldxMPHcBn/PozbC7uWUuTpR0nw4DjBvBacCM02Ub18='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_', 'replacing it with standby', 'No standby daemons available', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'wait-for-scrub': False, 'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 5}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': '8dc57b5624d909c1714a82ba84ff24e49431ea76', 'cluster': 'ceph'}}, {'cephfs_test_runner': {'modules': ['tasks.mgr.test_failover']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-29_15:35:32', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.11998'} 2022-11-29T18:03:25.671 DEBUG:teuthology.orchestra.run.smithi035:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-29T18:03:25.674 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-29T18:03:25.682 DEBUG:teuthology.orchestra.run.smithi035:> sudo install -d -m0777 -- /var/run/ceph 2022-11-29T18:03:25.724 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/run/ceph 2022-11-29T18:03:25.746 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:25.747 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/scratch_devs of=/dev/stdout 2022-11-29T18:03:25.759 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-29T18:03:25.760 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_1 2022-11-29T18:03:25.814 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-29T18:03:25.814 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-29T18:03:25.815 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 343 Links: 1 2022-11-29T18:03:25.815 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-29T18:03:25.816 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2022-11-29 18:00:46.683586831 +0000 2022-11-29T18:03:25.816 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2022-11-29 18:00:46.679586831 +0000 2022-11-29T18:03:25.816 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2022-11-29 18:00:46.679586831 +0000 2022-11-29T18:03:25.817 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2022-11-29T18:03:25.818 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-29T18:03:25.832 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2022-11-29T18:03:25.833 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2022-11-29T18:03:25.833 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.00040677 s, 1.3 MB/s 2022-11-29T18:03:25.835 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-29T18:03:25.888 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_2 2022-11-29T18:03:25.901 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-29T18:03:25.902 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-29T18:03:25.902 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 353 Links: 1 2022-11-29T18:03:25.902 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-29T18:03:25.903 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2022-11-29 18:00:47.263586818 +0000 2022-11-29T18:03:25.903 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2022-11-29 18:00:47.259586818 +0000 2022-11-29T18:03:25.904 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2022-11-29 18:00:47.259586818 +0000 2022-11-29T18:03:25.904 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2022-11-29T18:03:25.905 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-29T18:03:25.918 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2022-11-29T18:03:25.919 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2022-11-29T18:03:25.919 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000410688 s, 1.2 MB/s 2022-11-29T18:03:25.921 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-29T18:03:25.975 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_3 2022-11-29T18:03:25.983 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-29T18:03:25.984 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-29T18:03:25.984 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 358 Links: 1 2022-11-29T18:03:25.984 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-29T18:03:25.985 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2022-11-29 18:00:48.143586798 +0000 2022-11-29T18:03:25.985 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2022-11-29 18:00:47.827586805 +0000 2022-11-29T18:03:25.986 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2022-11-29 18:00:47.827586805 +0000 2022-11-29T18:03:25.986 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2022-11-29T18:03:25.987 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-29T18:03:26.001 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2022-11-29T18:03:26.002 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2022-11-29T18:03:26.002 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000358806 s, 1.4 MB/s 2022-11-29T18:03:26.003 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-29T18:03:26.012 DEBUG:teuthology.orchestra.run.smithi035:> stat /dev/vg_nvme/lv_4 2022-11-29T18:03:26.018 INFO:teuthology.orchestra.run.smithi035.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-29T18:03:26.018 INFO:teuthology.orchestra.run.smithi035.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-29T18:03:26.019 INFO:teuthology.orchestra.run.smithi035.stdout:Device: 6h/6d Inode: 366 Links: 1 2022-11-29T18:03:26.019 INFO:teuthology.orchestra.run.smithi035.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-29T18:03:26.019 INFO:teuthology.orchestra.run.smithi035.stdout:Access: 2022-11-29 18:00:48.399586793 +0000 2022-11-29T18:03:26.019 INFO:teuthology.orchestra.run.smithi035.stdout:Modify: 2022-11-29 18:00:48.395586793 +0000 2022-11-29T18:03:26.020 INFO:teuthology.orchestra.run.smithi035.stdout:Change: 2022-11-29 18:00:48.395586793 +0000 2022-11-29T18:03:26.020 INFO:teuthology.orchestra.run.smithi035.stdout: Birth: - 2022-11-29T18:03:26.020 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-29T18:03:26.077 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records in 2022-11-29T18:03:26.077 INFO:teuthology.orchestra.run.smithi035.stderr:1+0 records out 2022-11-29T18:03:26.077 INFO:teuthology.orchestra.run.smithi035.stderr:512 bytes copied, 0.000420527 s, 1.2 MB/s 2022-11-29T18:03:26.078 DEBUG:teuthology.orchestra.run.smithi035:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-29T18:03:26.088 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2022-11-29T18:03:26.088 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:26.089 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2022-11-29T18:03:26.097 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-29T18:03:26.097 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2022-11-29T18:03:26.147 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-29T18:03:26.147 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-29T18:03:26.148 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 359 Links: 1 2022-11-29T18:03:26.148 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-29T18:03:26.148 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2022-11-29 18:00:46.855253464 +0000 2022-11-29T18:03:26.148 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2022-11-29 18:00:46.535253468 +0000 2022-11-29T18:03:26.149 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2022-11-29 18:00:46.535253468 +0000 2022-11-29T18:03:26.149 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2022-11-29T18:03:26.149 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-29T18:03:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2022-11-29T18:03:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2022-11-29T18:03:26.161 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000428499 s, 1.2 MB/s 2022-11-29T18:03:26.162 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-29T18:03:26.213 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2022-11-29T18:03:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-29T18:03:26.221 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-29T18:03:26.222 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 367 Links: 1 2022-11-29T18:03:26.222 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-29T18:03:26.222 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2022-11-29 18:00:47.079253462 +0000 2022-11-29T18:03:26.222 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2022-11-29 18:00:47.075253462 +0000 2022-11-29T18:03:26.223 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2022-11-29 18:00:47.075253462 +0000 2022-11-29T18:03:26.223 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2022-11-29T18:03:26.223 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-29T18:03:26.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2022-11-29T18:03:26.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2022-11-29T18:03:26.239 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000560036 s, 914 kB/s 2022-11-29T18:03:26.240 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-29T18:03:26.293 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2022-11-29T18:03:26.301 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-29T18:03:26.301 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-29T18:03:26.302 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 373 Links: 1 2022-11-29T18:03:26.302 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-29T18:03:26.302 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2022-11-29 18:00:47.947253452 +0000 2022-11-29T18:03:26.302 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2022-11-29 18:00:47.639253456 +0000 2022-11-29T18:03:26.302 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2022-11-29 18:00:47.639253456 +0000 2022-11-29T18:03:26.303 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2022-11-29T18:03:26.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-29T18:03:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2022-11-29T18:03:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2022-11-29T18:03:26.318 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000485597 s, 1.1 MB/s 2022-11-29T18:03:26.320 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-29T18:03:26.328 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2022-11-29T18:03:26.379 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-29T18:03:26.379 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-29T18:03:26.380 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 6h/6d Inode: 379 Links: 1 2022-11-29T18:03:26.380 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-29T18:03:26.380 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2022-11-29 18:00:48.487253446 +0000 2022-11-29T18:03:26.380 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2022-11-29 18:00:48.159253450 +0000 2022-11-29T18:03:26.381 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2022-11-29 18:00:48.159253450 +0000 2022-11-29T18:03:26.381 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: - 2022-11-29T18:03:26.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-29T18:03:26.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2022-11-29T18:03:26.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2022-11-29T18:03:26.393 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.00046784 s, 1.1 MB/s 2022-11-29T18:03:26.395 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-29T18:03:26.445 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2022-11-29T18:03:26.446 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi035.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2022-11-29T18:03:26.446 INFO:tasks.ceph:Generating config... 2022-11-29T18:03:26.448 INFO:tasks.ceph:[client] debug client = 20 2022-11-29T18:03:26.448 INFO:tasks.ceph:[client] debug mgrc = 20 2022-11-29T18:03:26.448 INFO:tasks.ceph:[client] debug ms = 1 2022-11-29T18:03:26.449 INFO:tasks.ceph:[global] mon election default strategy = 3 2022-11-29T18:03:26.449 INFO:tasks.ceph:[mds] debug mgrc = 20 2022-11-29T18:03:26.449 INFO:tasks.ceph:[mgr] debug client = 20 2022-11-29T18:03:26.450 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-29T18:03:26.450 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-29T18:03:26.451 INFO:tasks.ceph:[mgr] mgr ttl cache expire seconds = 5 2022-11-29T18:03:26.451 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-29T18:03:26.451 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-29T18:03:26.451 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-29T18:03:26.452 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2022-11-29T18:03:26.452 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2022-11-29T18:03:26.452 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-29T18:03:26.453 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-29T18:03:26.453 INFO:tasks.ceph:[osd] debug bluefs = 20 2022-11-29T18:03:26.453 INFO:tasks.ceph:[osd] debug bluestore = 20 2022-11-29T18:03:26.454 INFO:tasks.ceph:[osd] debug mgrc = 20 2022-11-29T18:03:26.454 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-29T18:03:26.454 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-29T18:03:26.455 INFO:tasks.ceph:[osd] debug rocksdb = 10 2022-11-29T18:03:26.455 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-29T18:03:26.455 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-29T18:03:26.456 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-29T18:03:26.456 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-11-29T18:03:26.456 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-11-29T18:03:26.457 INFO:tasks.ceph:Setting up mon.a... 2022-11-29T18:03:26.457 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-29T18:03:26.510 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.keyring 2022-11-29T18:03:26.513 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-29T18:03:26.547 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-29T18:03:26.562 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.35'), ('mon.c', '[v2:172.21.15.35:3301,v1:172.21.15.35:6790]'), ('mon.b', '172.21.15.38')] 2022-11-29T18:03:26.563 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.35,[v2:172.21.15.35:3301,v1:172.21.15.35:6790],172.21.15.38', 'mon election default strategy': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'mgr ttl cache expire seconds': 5}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds': {'debug mgrc': 20}} 2022-11-29T18:03:26.564 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:26.565 DEBUG:teuthology.orchestra.run.smithi035:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-29T18:03:26.612 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.35 --addv c '[v2:172.21.15.35:3301,v1:172.21.15.35:6790]' --add b 172.21.15.38 --print /home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:26.651 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:26.651 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: generated fsid 338b2b5b-2e59-4e89-9321-f2b909af04fd 2022-11-29T18:03:26.652 INFO:teuthology.orchestra.run.smithi035.stdout:epoch 0 2022-11-29T18:03:26.652 INFO:teuthology.orchestra.run.smithi035.stdout:fsid 338b2b5b-2e59-4e89-9321-f2b909af04fd 2022-11-29T18:03:26.653 INFO:teuthology.orchestra.run.smithi035.stdout:last_changed 2022-11-29T18:03:26.652191+0000 2022-11-29T18:03:26.653 INFO:teuthology.orchestra.run.smithi035.stdout:created 2022-11-29T18:03:26.652191+0000 2022-11-29T18:03:26.654 INFO:teuthology.orchestra.run.smithi035.stdout:min_mon_release 0 (unknown) 2022-11-29T18:03:26.654 INFO:teuthology.orchestra.run.smithi035.stdout:election_strategy: 3 2022-11-29T18:03:26.654 INFO:teuthology.orchestra.run.smithi035.stdout:0: [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] mon.a 2022-11-29T18:03:26.655 INFO:teuthology.orchestra.run.smithi035.stdout:1: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.b 2022-11-29T18:03:26.686 INFO:teuthology.orchestra.run.smithi035.stdout:2: [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] mon.c 2022-11-29T18:03:26.686 INFO:teuthology.orchestra.run.smithi035.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-29T18:03:26.697 DEBUG:teuthology.orchestra.run.smithi035:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-29T18:03:26.706 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 338b2b5b-2e59-4e89-9321-f2b909af04fd... 2022-11-29T18:03:26.708 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-29T18:03:26.751 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-29T18:03:26.783 INFO:teuthology.orchestra.run.smithi035.stdout:[global] 2022-11-29T18:03:26.783 INFO:teuthology.orchestra.run.smithi035.stdout: chdir = "" 2022-11-29T18:03:26.784 INFO:teuthology.orchestra.run.smithi035.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-29T18:03:26.784 INFO:teuthology.orchestra.run.smithi035.stdout: auth supported = cephx 2022-11-29T18:03:26.784 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.785 INFO:teuthology.orchestra.run.smithi035.stdout: filestore xattr use omap = true 2022-11-29T18:03:26.785 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.785 INFO:teuthology.orchestra.run.smithi035.stdout: mon clock drift allowed = 1.000 2022-11-29T18:03:26.785 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.786 INFO:teuthology.orchestra.run.smithi035.stdout: osd crush chooseleaf type = 0 2022-11-29T18:03:26.786 INFO:teuthology.orchestra.run.smithi035.stdout: auth debug = true 2022-11-29T18:03:26.786 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.786 INFO:teuthology.orchestra.run.smithi035.stdout: ms die on old message = true 2022-11-29T18:03:26.787 INFO:teuthology.orchestra.run.smithi035.stdout: ms die on bug = true 2022-11-29T18:03:26.787 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.787 INFO:teuthology.orchestra.run.smithi035.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-29T18:03:26.787 INFO:teuthology.orchestra.run.smithi035.stdout: mon pg warn max object skew = 0 2022-11-29T18:03:26.788 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.788 INFO:teuthology.orchestra.run.smithi035.stdout: # disable pg_autoscaler by default for new pools 2022-11-29T18:03:26.788 INFO:teuthology.orchestra.run.smithi035.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-29T18:03:26.789 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.789 INFO:teuthology.orchestra.run.smithi035.stdout: osd pool default size = 2 2022-11-29T18:03:26.789 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.790 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd allow primary affinity = true 2022-11-29T18:03:26.790 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd allow pg remap = true 2022-11-29T18:03:26.791 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on legacy crush tunables = false 2022-11-29T18:03:26.791 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on crush straw calc version zero = false 2022-11-29T18:03:26.791 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on no sortbitwise = false 2022-11-29T18:03:26.792 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on osd down out interval zero = false 2022-11-29T18:03:26.792 INFO:teuthology.orchestra.run.smithi035.stdout: mon warn on too few osds = false 2022-11-29T18:03:26.792 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-29T18:03:26.793 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-29T18:03:26.793 INFO:teuthology.orchestra.run.smithi035.stdout: mon_allow_pool_size_one = true 2022-11-29T18:03:26.793 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.794 INFO:teuthology.orchestra.run.smithi035.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-11-29T18:03:26.794 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.794 INFO:teuthology.orchestra.run.smithi035.stdout: osd default data pool replay window = 5 2022-11-29T18:03:26.795 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.795 INFO:teuthology.orchestra.run.smithi035.stdout: mon allow pool delete = true 2022-11-29T18:03:26.796 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.796 INFO:teuthology.orchestra.run.smithi035.stdout: mon cluster log file level = debug 2022-11-29T18:03:26.797 INFO:teuthology.orchestra.run.smithi035.stdout: debug asserts on shutdown = true 2022-11-29T18:03:26.797 INFO:teuthology.orchestra.run.smithi035.stdout: mon health detail to clog = false 2022-11-29T18:03:26.798 INFO:teuthology.orchestra.run.smithi035.stdout: mon host = "172.21.15.35,[v2:172.21.15.35:3301,v1:172.21.15.35:6790],172.21.15.38" 2022-11-29T18:03:26.798 INFO:teuthology.orchestra.run.smithi035.stdout: mon election default strategy = 3 2022-11-29T18:03:26.798 INFO:teuthology.orchestra.run.smithi035.stdout: fsid = 338b2b5b-2e59-4e89-9321-f2b909af04fd 2022-11-29T18:03:26.799 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.799 INFO:teuthology.orchestra.run.smithi035.stdout:[osd] 2022-11-29T18:03:26.800 INFO:teuthology.orchestra.run.smithi035.stdout: osd journal size = 100 2022-11-29T18:03:26.800 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.801 INFO:teuthology.orchestra.run.smithi035.stdout: osd scrub load threshold = 5.0 2022-11-29T18:03:26.801 INFO:teuthology.orchestra.run.smithi035.stdout: osd scrub max interval = 600 2022-11-29T18:03:26.802 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.802 INFO:teuthology.orchestra.run.smithi035.stdout: osd recover clone overlap = true 2022-11-29T18:03:26.803 INFO:teuthology.orchestra.run.smithi035.stdout: osd recovery max chunk = 1048576 2022-11-29T18:03:26.803 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.804 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug shutdown = true 2022-11-29T18:03:26.804 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug op order = true 2022-11-29T18:03:26.805 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug verify stray on activate = true 2022-11-29T18:03:26.805 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.806 INFO:teuthology.orchestra.run.smithi035.stdout: osd open classes on start = true 2022-11-29T18:03:26.806 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug pg log writeout = true 2022-11-29T18:03:26.807 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.807 INFO:teuthology.orchestra.run.smithi035.stdout: osd deep scrub update digest min age = 30 2022-11-29T18:03:26.807 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.808 INFO:teuthology.orchestra.run.smithi035.stdout: osd map max advance = 10 2022-11-29T18:03:26.808 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.809 INFO:teuthology.orchestra.run.smithi035.stdout: journal zero on create = true 2022-11-29T18:03:26.809 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.810 INFO:teuthology.orchestra.run.smithi035.stdout: filestore ondisk finisher threads = 3 2022-11-29T18:03:26.810 INFO:teuthology.orchestra.run.smithi035.stdout: filestore apply finisher threads = 3 2022-11-29T18:03:26.811 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.811 INFO:teuthology.orchestra.run.smithi035.stdout: bdev debug aio = true 2022-11-29T18:03:26.811 INFO:teuthology.orchestra.run.smithi035.stdout: osd debug misdirected ops = true 2022-11-29T18:03:26.812 INFO:teuthology.orchestra.run.smithi035.stdout: bluefs allocator = hybrid 2022-11-29T18:03:26.812 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore allocator = hybrid 2022-11-29T18:03:26.813 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore block size = 96636764160 2022-11-29T18:03:26.813 INFO:teuthology.orchestra.run.smithi035.stdout: bluestore fsck on mount = True 2022-11-29T18:03:26.813 INFO:teuthology.orchestra.run.smithi035.stdout: debug bluefs = 20 2022-11-29T18:03:26.814 INFO:teuthology.orchestra.run.smithi035.stdout: debug bluestore = 20 2022-11-29T18:03:26.814 INFO:teuthology.orchestra.run.smithi035.stdout: debug mgrc = 20 2022-11-29T18:03:26.815 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-29T18:03:26.815 INFO:teuthology.orchestra.run.smithi035.stdout: debug osd = 20 2022-11-29T18:03:26.816 INFO:teuthology.orchestra.run.smithi035.stdout: debug rocksdb = 10 2022-11-29T18:03:26.816 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd backfillfull_ratio = 0.85 2022-11-29T18:03:26.817 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd full ratio = 0.9 2022-11-29T18:03:26.817 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd nearfull ratio = 0.8 2022-11-29T18:03:26.818 INFO:teuthology.orchestra.run.smithi035.stdout: osd failsafe full ratio = 0.95 2022-11-29T18:03:26.818 INFO:teuthology.orchestra.run.smithi035.stdout: osd objectstore = bluestore 2022-11-29T18:03:26.818 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.819 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr] 2022-11-29T18:03:26.819 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-29T18:03:26.820 INFO:teuthology.orchestra.run.smithi035.stdout: debug mgr = 20 2022-11-29T18:03:26.820 INFO:teuthology.orchestra.run.smithi035.stdout: debug mon = 20 2022-11-29T18:03:26.821 INFO:teuthology.orchestra.run.smithi035.stdout: debug auth = 20 2022-11-29T18:03:26.821 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min pgs per osd = 4 2022-11-29T18:03:26.822 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min bytes per osd = 10 2022-11-29T18:03:26.822 INFO:teuthology.orchestra.run.smithi035.stdout: mgr/telemetry/nag = false 2022-11-29T18:03:26.823 INFO:teuthology.orchestra.run.smithi035.stdout: debug client = 20 2022-11-29T18:03:26.823 INFO:teuthology.orchestra.run.smithi035.stdout: mgr ttl cache expire seconds = 5 2022-11-29T18:03:26.824 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.824 INFO:teuthology.orchestra.run.smithi035.stdout:[mon] 2022-11-29T18:03:26.825 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-29T18:03:26.825 INFO:teuthology.orchestra.run.smithi035.stdout: debug mon = 20 2022-11-29T18:03:26.826 INFO:teuthology.orchestra.run.smithi035.stdout: debug paxos = 20 2022-11-29T18:03:26.826 INFO:teuthology.orchestra.run.smithi035.stdout: debug auth = 20 2022-11-29T18:03:26.827 INFO:teuthology.orchestra.run.smithi035.stdout: mon data avail warn = 5 2022-11-29T18:03:26.827 INFO:teuthology.orchestra.run.smithi035.stdout: mon mgr mkfs grace = 240 2022-11-29T18:03:26.827 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min pgs per osd = 4 2022-11-29T18:03:26.828 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd reporter subtree level = osd 2022-11-29T18:03:26.828 INFO:teuthology.orchestra.run.smithi035.stdout: mon osd prime pg temp = true 2022-11-29T18:03:26.829 INFO:teuthology.orchestra.run.smithi035.stdout: mon reweight min bytes per osd = 10 2022-11-29T18:03:26.829 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.829 INFO:teuthology.orchestra.run.smithi035.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-29T18:03:26.830 INFO:teuthology.orchestra.run.smithi035.stdout: auth mon ticket ttl = 660 # 11m 2022-11-29T18:03:26.830 INFO:teuthology.orchestra.run.smithi035.stdout: auth service ticket ttl = 240 # 4m 2022-11-29T18:03:26.830 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.831 INFO:teuthology.orchestra.run.smithi035.stdout: # don't complain about insecure global_id in the test suite 2022-11-29T18:03:26.831 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-29T18:03:26.832 INFO:teuthology.orchestra.run.smithi035.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-29T18:03:26.832 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:26.832 INFO:teuthology.orchestra.run.smithi035.stdout:[client] 2022-11-29T18:03:26.833 INFO:teuthology.orchestra.run.smithi035.stdout: rgw cache enabled = true 2022-11-29T18:03:26.833 INFO:teuthology.orchestra.run.smithi035.stdout: rgw enable ops log = true 2022-11-29T18:03:26.833 INFO:teuthology.orchestra.run.smithi035.stdout: rgw enable usage log = true 2022-11-29T18:03:26.833 INFO:teuthology.orchestra.run.smithi035.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-29T18:03:26.833 INFO:teuthology.orchestra.run.smithi035.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-29T18:03:26.834 INFO:teuthology.orchestra.run.smithi035.stdout: debug client = 20 2022-11-29T18:03:26.834 INFO:teuthology.orchestra.run.smithi035.stdout: debug mgrc = 20 2022-11-29T18:03:26.834 INFO:teuthology.orchestra.run.smithi035.stdout: debug ms = 1 2022-11-29T18:03:26.834 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.a] 2022-11-29T18:03:26.835 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.c] 2022-11-29T18:03:26.835 INFO:teuthology.orchestra.run.smithi035.stdout:[mon.b] 2022-11-29T18:03:26.835 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.a] 2022-11-29T18:03:26.835 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.c] 2022-11-29T18:03:26.835 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.b] 2022-11-29T18:03:26.836 INFO:teuthology.orchestra.run.smithi035.stdout:[mds] 2022-11-29T18:03:26.836 INFO:teuthology.orchestra.run.smithi035.stdout: debug mgrc = 20 2022-11-29T18:03:26.838 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2022-11-29T18:03:26.838 INFO:teuthology.orchestra.run.smithi038.stdout: chdir = "" 2022-11-29T18:03:26.839 INFO:teuthology.orchestra.run.smithi038.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-29T18:03:26.839 INFO:teuthology.orchestra.run.smithi038.stdout: auth supported = cephx 2022-11-29T18:03:26.839 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.839 INFO:teuthology.orchestra.run.smithi038.stdout: filestore xattr use omap = true 2022-11-29T18:03:26.840 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.840 INFO:teuthology.orchestra.run.smithi038.stdout: mon clock drift allowed = 1.000 2022-11-29T18:03:26.840 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.840 INFO:teuthology.orchestra.run.smithi038.stdout: osd crush chooseleaf type = 0 2022-11-29T18:03:26.841 INFO:teuthology.orchestra.run.smithi038.stdout: auth debug = true 2022-11-29T18:03:26.841 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.841 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on old message = true 2022-11-29T18:03:26.842 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on bug = true 2022-11-29T18:03:26.842 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.842 INFO:teuthology.orchestra.run.smithi038.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-29T18:03:26.842 INFO:teuthology.orchestra.run.smithi038.stdout: mon pg warn max object skew = 0 2022-11-29T18:03:26.842 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.843 INFO:teuthology.orchestra.run.smithi038.stdout: # disable pg_autoscaler by default for new pools 2022-11-29T18:03:26.843 INFO:teuthology.orchestra.run.smithi038.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-29T18:03:26.843 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.843 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default size = 2 2022-11-29T18:03:26.844 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.844 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow primary affinity = true 2022-11-29T18:03:26.844 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow pg remap = true 2022-11-29T18:03:26.844 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on legacy crush tunables = false 2022-11-29T18:03:26.845 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on crush straw calc version zero = false 2022-11-29T18:03:26.845 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on no sortbitwise = false 2022-11-29T18:03:26.845 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on osd down out interval zero = false 2022-11-29T18:03:26.845 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on too few osds = false 2022-11-29T18:03:26.845 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-29T18:03:26.846 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-29T18:03:26.846 INFO:teuthology.orchestra.run.smithi038.stdout: mon_allow_pool_size_one = true 2022-11-29T18:03:26.846 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.846 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2022-11-29T18:03:26.847 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.847 INFO:teuthology.orchestra.run.smithi038.stdout: osd default data pool replay window = 5 2022-11-29T18:03:26.847 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.847 INFO:teuthology.orchestra.run.smithi038.stdout: mon allow pool delete = true 2022-11-29T18:03:26.848 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.848 INFO:teuthology.orchestra.run.smithi038.stdout: mon cluster log file level = debug 2022-11-29T18:03:26.848 INFO:teuthology.orchestra.run.smithi038.stdout: debug asserts on shutdown = true 2022-11-29T18:03:26.848 INFO:teuthology.orchestra.run.smithi038.stdout: mon health detail to clog = false 2022-11-29T18:03:26.849 INFO:teuthology.orchestra.run.smithi038.stdout: mon host = "172.21.15.35,[v2:172.21.15.35:3301,v1:172.21.15.35:6790],172.21.15.38" 2022-11-29T18:03:26.849 INFO:teuthology.orchestra.run.smithi038.stdout: mon election default strategy = 3 2022-11-29T18:03:26.849 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 338b2b5b-2e59-4e89-9321-f2b909af04fd 2022-11-29T18:03:26.849 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.849 INFO:teuthology.orchestra.run.smithi038.stdout:[osd] 2022-11-29T18:03:26.850 INFO:teuthology.orchestra.run.smithi038.stdout: osd journal size = 100 2022-11-29T18:03:26.850 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.850 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub load threshold = 5.0 2022-11-29T18:03:26.850 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub max interval = 600 2022-11-29T18:03:26.850 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.851 INFO:teuthology.orchestra.run.smithi038.stdout: osd recover clone overlap = true 2022-11-29T18:03:26.851 INFO:teuthology.orchestra.run.smithi038.stdout: osd recovery max chunk = 1048576 2022-11-29T18:03:26.851 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.851 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug shutdown = true 2022-11-29T18:03:26.852 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug op order = true 2022-11-29T18:03:26.852 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug verify stray on activate = true 2022-11-29T18:03:26.852 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.852 INFO:teuthology.orchestra.run.smithi038.stdout: osd open classes on start = true 2022-11-29T18:03:26.853 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug pg log writeout = true 2022-11-29T18:03:26.853 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.853 INFO:teuthology.orchestra.run.smithi038.stdout: osd deep scrub update digest min age = 30 2022-11-29T18:03:26.853 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.853 INFO:teuthology.orchestra.run.smithi038.stdout: osd map max advance = 10 2022-11-29T18:03:26.854 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.854 INFO:teuthology.orchestra.run.smithi038.stdout: journal zero on create = true 2022-11-29T18:03:26.854 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.854 INFO:teuthology.orchestra.run.smithi038.stdout: filestore ondisk finisher threads = 3 2022-11-29T18:03:26.854 INFO:teuthology.orchestra.run.smithi038.stdout: filestore apply finisher threads = 3 2022-11-29T18:03:26.855 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.855 INFO:teuthology.orchestra.run.smithi038.stdout: bdev debug aio = true 2022-11-29T18:03:26.855 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug misdirected ops = true 2022-11-29T18:03:26.856 INFO:teuthology.orchestra.run.smithi038.stdout: bluefs allocator = hybrid 2022-11-29T18:03:26.856 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore allocator = hybrid 2022-11-29T18:03:26.856 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore block size = 96636764160 2022-11-29T18:03:26.857 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore fsck on mount = True 2022-11-29T18:03:26.857 INFO:teuthology.orchestra.run.smithi038.stdout: debug bluefs = 20 2022-11-29T18:03:26.858 INFO:teuthology.orchestra.run.smithi038.stdout: debug bluestore = 20 2022-11-29T18:03:26.858 INFO:teuthology.orchestra.run.smithi038.stdout: debug mgrc = 20 2022-11-29T18:03:26.859 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2022-11-29T18:03:26.859 INFO:teuthology.orchestra.run.smithi038.stdout: debug osd = 20 2022-11-29T18:03:26.859 INFO:teuthology.orchestra.run.smithi038.stdout: debug rocksdb = 10 2022-11-29T18:03:26.860 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd backfillfull_ratio = 0.85 2022-11-29T18:03:26.860 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd full ratio = 0.9 2022-11-29T18:03:26.861 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd nearfull ratio = 0.8 2022-11-29T18:03:26.861 INFO:teuthology.orchestra.run.smithi038.stdout: osd failsafe full ratio = 0.95 2022-11-29T18:03:26.862 INFO:teuthology.orchestra.run.smithi038.stdout: osd objectstore = bluestore 2022-11-29T18:03:26.862 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.862 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr] 2022-11-29T18:03:26.862 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2022-11-29T18:03:26.863 INFO:teuthology.orchestra.run.smithi038.stdout: debug mgr = 20 2022-11-29T18:03:26.863 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2022-11-29T18:03:26.863 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2022-11-29T18:03:26.863 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2022-11-29T18:03:26.863 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2022-11-29T18:03:26.864 INFO:teuthology.orchestra.run.smithi038.stdout: mgr/telemetry/nag = false 2022-11-29T18:03:26.864 INFO:teuthology.orchestra.run.smithi038.stdout: debug client = 20 2022-11-29T18:03:26.864 INFO:teuthology.orchestra.run.smithi038.stdout: mgr ttl cache expire seconds = 5 2022-11-29T18:03:26.864 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.865 INFO:teuthology.orchestra.run.smithi038.stdout:[mon] 2022-11-29T18:03:26.865 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2022-11-29T18:03:26.865 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2022-11-29T18:03:26.865 INFO:teuthology.orchestra.run.smithi038.stdout: debug paxos = 20 2022-11-29T18:03:26.865 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2022-11-29T18:03:26.866 INFO:teuthology.orchestra.run.smithi038.stdout: mon data avail warn = 5 2022-11-29T18:03:26.866 INFO:teuthology.orchestra.run.smithi038.stdout: mon mgr mkfs grace = 240 2022-11-29T18:03:26.866 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2022-11-29T18:03:26.866 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd reporter subtree level = osd 2022-11-29T18:03:26.867 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd prime pg temp = true 2022-11-29T18:03:26.867 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2022-11-29T18:03:26.867 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.867 INFO:teuthology.orchestra.run.smithi038.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-29T18:03:26.867 INFO:teuthology.orchestra.run.smithi038.stdout: auth mon ticket ttl = 660 # 11m 2022-11-29T18:03:26.868 INFO:teuthology.orchestra.run.smithi038.stdout: auth service ticket ttl = 240 # 4m 2022-11-29T18:03:26.868 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.868 INFO:teuthology.orchestra.run.smithi038.stdout: # don't complain about insecure global_id in the test suite 2022-11-29T18:03:26.868 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-29T18:03:26.869 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-29T18:03:26.869 INFO:teuthology.orchestra.run.smithi038.stdout: 2022-11-29T18:03:26.869 INFO:teuthology.orchestra.run.smithi038.stdout:[client] 2022-11-29T18:03:26.870 INFO:teuthology.orchestra.run.smithi038.stdout: rgw cache enabled = true 2022-11-29T18:03:26.870 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable ops log = true 2022-11-29T18:03:26.870 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable usage log = true 2022-11-29T18:03:26.870 INFO:teuthology.orchestra.run.smithi038.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-29T18:03:26.871 INFO:teuthology.orchestra.run.smithi038.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-29T18:03:26.871 INFO:teuthology.orchestra.run.smithi038.stdout: debug client = 20 2022-11-29T18:03:26.871 INFO:teuthology.orchestra.run.smithi038.stdout: debug mgrc = 20 2022-11-29T18:03:26.871 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2022-11-29T18:03:26.871 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.a] 2022-11-29T18:03:26.872 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.c] 2022-11-29T18:03:26.872 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.b] 2022-11-29T18:03:26.872 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2022-11-29T18:03:26.872 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c] 2022-11-29T18:03:26.873 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b] 2022-11-29T18:03:26.873 INFO:teuthology.orchestra.run.smithi038.stdout:[mds] 2022-11-29T18:03:26.873 INFO:teuthology.orchestra.run.smithi038.stdout: debug mgrc = 20 2022-11-29T18:03:26.874 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-29T18:03:26.874 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-29T18:03:26.923 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-29T18:03:26.924 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:26.924 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-29T18:03:26.930 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:26.931 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-29T18:03:26.938 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi035.front.sepia.ceph.com 2022-11-29T18:03:26.938 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:26.939 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-29T18:03:26.939 DEBUG:teuthology.orchestra.run.smithi035:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-29T18:03:27.009 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:27.010 DEBUG:teuthology.orchestra.run.smithi035:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:27.016 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi038.front.sepia.ceph.com 2022-11-29T18:03:27.017 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:27.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-29T18:03:27.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-29T18:03:27.045 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:27.046 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:27.094 INFO:tasks.ceph:Setting up mon nodes... 2022-11-29T18:03:27.095 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-29T18:03:27.095 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-29T18:03:27.138 INFO:teuthology.orchestra.run.smithi035.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-29T18:03:27.141 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-29T18:03:27.207 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-29T18:03:27.210 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-z && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.z /var/lib/ceph/mgr/ceph-z/keyring 2022-11-29T18:03:27.253 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mgr/ceph-z/keyring 2022-11-29T18:03:27.256 INFO:tasks.ceph:Setting up mds nodes... 2022-11-29T18:03:27.256 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-29T18:03:27.300 INFO:teuthology.orchestra.run.smithi035.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-29T18:03:27.302 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-29T18:03:27.313 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-29T18:03:27.358 INFO:teuthology.orchestra.run.smithi035.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-29T18:03:27.361 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-29T18:03:27.372 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-29T18:03:27.416 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-29T18:03:27.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-29T18:03:27.476 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-29T18:03:27.477 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-29T18:03:27.514 INFO:teuthology.orchestra.run.smithi035.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-29T18:03:27.528 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-29T18:03:27.565 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-29T18:03:27.578 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-29T18:03:27.579 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi035.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2022-11-29T18:03:27.579 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-29T18:03:27.597 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2022-11-29T18:03:27.598 INFO:tasks.ceph:role: osd.0 2022-11-29T18:03:27.599 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi035.front.sepia.ceph.com 2022-11-29T18:03:27.599 DEBUG:teuthology.orchestra.run.smithi035:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-29T18:03:28.129 INFO:teuthology.orchestra.run.smithi035.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-29T18:03:28.130 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-29T18:03:28.130 INFO:teuthology.orchestra.run.smithi035.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-29T18:03:28.130 INFO:teuthology.orchestra.run.smithi035.stdout: = reflink=1 2022-11-29T18:03:28.130 INFO:teuthology.orchestra.run.smithi035.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-29T18:03:28.131 INFO:teuthology.orchestra.run.smithi035.stdout: = sunit=0 swidth=0 blks 2022-11-29T18:03:28.131 INFO:teuthology.orchestra.run.smithi035.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-29T18:03:28.131 INFO:teuthology.orchestra.run.smithi035.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-29T18:03:28.131 INFO:teuthology.orchestra.run.smithi035.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-29T18:03:28.131 INFO:teuthology.orchestra.run.smithi035.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-29T18:03:28.133 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi035.front.sepia.ceph.com -o noatime 2022-11-29T18:03:28.133 DEBUG:teuthology.orchestra.run.smithi035:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-29T18:03:28.182 DEBUG:teuthology.orchestra.run.smithi035:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-29T18:03:28.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-29T18:03:28.194 INFO:teuthology.orchestra.run.smithi035.stderr:sudo: /sbin/restorecon: command not found 2022-11-29T18:03:28.195 DEBUG:teuthology.orchestra.run.smithi035:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:28.262 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:28.262+0000 7fd956b48080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-29T18:03:28.265 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:28.262+0000 7fd956b48080 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-29T18:03:28.265 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:28.262+0000 7fd956b48080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-29T18:03:28.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:28.262+0000 7fd956b48080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-29T18:03:28.266 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:28.262+0000 7fd956b48080 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-29T18:03:28.269 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:28.266+0000 7fd956b48080 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-11-29T18:03:31.234 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-29T18:03:31.252 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-29T18:03:31.271 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2022-11-29T18:03:31.272 INFO:tasks.ceph:role: osd.1 2022-11-29T18:03:31.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com 2022-11-29T18:03:31.272 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-29T18:03:32.614 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-29T18:03:32.614 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-29T18:03:32.614 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-29T18:03:32.615 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2022-11-29T18:03:32.615 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-29T18:03:32.615 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2022-11-29T18:03:32.616 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-29T18:03:32.616 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-29T18:03:32.616 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-29T18:03:32.617 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-29T18:03:32.618 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2022-11-29T18:03:32.619 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2022-11-29T18:03:32.645 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-29T18:03:32.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-29T18:03:32.656 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: /sbin/restorecon: command not found 2022-11-29T18:03:32.656 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-29T18:03:32.669 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2022-11-29T18:03:32.669 INFO:tasks.ceph:role: osd.2 2022-11-29T18:03:32.669 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com 2022-11-29T18:03:32.670 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-29T18:03:34.061 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-29T18:03:34.062 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-29T18:03:34.062 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-29T18:03:34.062 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2022-11-29T18:03:34.063 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-29T18:03:34.063 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2022-11-29T18:03:34.064 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-29T18:03:34.064 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-29T18:03:34.065 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-29T18:03:34.065 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-29T18:03:34.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2022-11-29T18:03:34.068 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2022-11-29T18:03:34.096 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-29T18:03:34.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-29T18:03:34.153 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: /sbin/restorecon: command not found 2022-11-29T18:03:34.153 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-29T18:03:34.167 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2022-11-29T18:03:34.167 INFO:tasks.ceph:role: osd.3 2022-11-29T18:03:34.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi038.front.sepia.ceph.com 2022-11-29T18:03:34.169 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-29T18:03:35.516 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-29T18:03:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-29T18:03:35.517 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-29T18:03:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 2022-11-29T18:03:35.518 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-29T18:03:35.518 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2022-11-29T18:03:35.519 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-29T18:03:35.519 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-29T18:03:35.520 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-29T18:03:35.520 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-29T18:03:35.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2022-11-29T18:03:35.523 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2022-11-29T18:03:35.552 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-29T18:03:35.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-29T18:03:35.562 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: /sbin/restorecon: command not found 2022-11-29T18:03:35.563 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:35.634 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:35.629+0000 7fb076959080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-29T18:03:35.634 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:35.629+0000 7fb076959080 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-29T18:03:35.635 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:35.629+0000 7fb076959080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-29T18:03:35.635 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:35.629+0000 7fb076959080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-29T18:03:35.635 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:35.629+0000 7fb076959080 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-29T18:03:35.636 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:35.629+0000 7fb076959080 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-11-29T18:03:38.656 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-29T18:03:38.673 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:38.712 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:38.705+0000 7f1ac9d0c080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-29T18:03:38.713 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:38.709+0000 7f1ac9d0c080 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-29T18:03:38.713 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:38.709+0000 7f1ac9d0c080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-29T18:03:38.713 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:38.709+0000 7f1ac9d0c080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-29T18:03:38.713 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:38.709+0000 7f1ac9d0c080 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-29T18:03:38.714 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:38.709+0000 7f1ac9d0c080 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-11-29T18:03:41.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-29T18:03:41.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:41.737+0000 7f54c46b2080 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-29T18:03:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:41.737+0000 7f54c46b2080 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-29T18:03:41.745 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:41.737+0000 7f54c46b2080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-29T18:03:41.745 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:41.737+0000 7f54c46b2080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-29T18:03:41.746 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:41.737+0000 7f54c46b2080 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-29T18:03:41.747 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:41.741+0000 7f54c46b2080 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-11-29T18:03:44.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-29T18:03:44.772 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-29T18:03:44.773 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:44.774 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-29T18:03:44.792 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:44.793 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-29T18:03:44.855 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:44.856 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-29T18:03:44.870 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:44.871 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-29T18:03:44.928 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:44.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-29T18:03:44.946 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:44.947 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mgr/ceph-z/keyring of=/dev/stdout 2022-11-29T18:03:44.958 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:44.958 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-29T18:03:45.015 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:45.016 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-29T18:03:45.030 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:45.030 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-29T18:03:45.093 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:45.094 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-29T18:03:45.107 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:45.107 DEBUG:teuthology.orchestra.run.smithi035:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-29T18:03:45.116 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:45.117 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-29T18:03:45.154 INFO:tasks.ceph:Adding keys to all mons... 2022-11-29T18:03:45.155 DEBUG:teuthology.orchestra.run.smithi035:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-29T18:03:45.159 DEBUG:teuthology.orchestra.run.smithi038:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-29T18:03:45.171 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.x] 2022-11-29T18:03:45.172 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBvSYZj5jJUCBAAFwH4fcSKllneEvOAFKjvFA== 2022-11-29T18:03:45.172 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.a] 2022-11-29T18:03:45.173 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBvSYZjnXP9ERAA5fX354lnEJ/8osrGl1SaBw== 2022-11-29T18:03:45.173 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.c] 2022-11-29T18:03:45.174 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBvSYZj/1l1FRAAZWOnEuDji1txljX/m8WiHw== 2022-11-29T18:03:45.174 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.0] 2022-11-29T18:03:45.174 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBwSYZjeUW7DxAAErvQRbRaOUseX2h0FtwNxw== 2022-11-29T18:03:45.175 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.y] 2022-11-29T18:03:45.175 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBvSYZjE2tbDBAA5M/dKVa5IVvuEP8sXAqrYQ== 2022-11-29T18:03:45.175 INFO:teuthology.orchestra.run.smithi035.stdout:[mgr.z] 2022-11-29T18:03:45.176 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBvSYZjXUcfDxAA8sxex/MObgxiDA291CFJMA== 2022-11-29T18:03:45.176 INFO:teuthology.orchestra.run.smithi035.stdout:[mds.b] 2022-11-29T18:03:45.176 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBvSYZjmqjYGBAAECQa4ipP4IgJpeQTpJXczw== 2022-11-29T18:03:45.176 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.1] 2022-11-29T18:03:45.176 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQB3SYZjMLnKJRAAQTNcsx5zYcJl4vdyrJ2Oww== 2022-11-29T18:03:45.177 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.2] 2022-11-29T18:03:45.177 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQB6SYZjj/d5KhAAWJNxD6rW2pRTioqAkWcBsg== 2022-11-29T18:03:45.178 INFO:teuthology.orchestra.run.smithi035.stdout:[osd.3] 2022-11-29T18:03:45.178 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQB9SYZjy7pbLBAAAi/cqyJyeosulwMzY6R+ew== 2022-11-29T18:03:45.178 INFO:teuthology.orchestra.run.smithi035.stdout:[client.0] 2022-11-29T18:03:45.178 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBvSYZj1bS8HhAA7dfrTRaGOeNG5riROdPYsQ== 2022-11-29T18:03:45.179 INFO:teuthology.orchestra.run.smithi035.stdout:[client.1] 2022-11-29T18:03:45.179 INFO:teuthology.orchestra.run.smithi035.stdout: key = AQBvSYZj86C1IRAASCCV5uafbadEwOwUEEOGmA== 2022-11-29T18:03:45.181 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.x] 2022-11-29T18:03:45.181 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBvSYZj5jJUCBAAFwH4fcSKllneEvOAFKjvFA== 2022-11-29T18:03:45.181 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2022-11-29T18:03:45.181 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBvSYZjnXP9ERAA5fX354lnEJ/8osrGl1SaBw== 2022-11-29T18:03:45.182 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c] 2022-11-29T18:03:45.182 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBvSYZj/1l1FRAAZWOnEuDji1txljX/m8WiHw== 2022-11-29T18:03:45.182 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.0] 2022-11-29T18:03:45.182 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBwSYZjeUW7DxAAErvQRbRaOUseX2h0FtwNxw== 2022-11-29T18:03:45.182 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.y] 2022-11-29T18:03:45.183 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBvSYZjE2tbDBAA5M/dKVa5IVvuEP8sXAqrYQ== 2022-11-29T18:03:45.183 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.z] 2022-11-29T18:03:45.183 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBvSYZjXUcfDxAA8sxex/MObgxiDA291CFJMA== 2022-11-29T18:03:45.183 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b] 2022-11-29T18:03:45.184 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBvSYZjmqjYGBAAECQa4ipP4IgJpeQTpJXczw== 2022-11-29T18:03:45.184 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.1] 2022-11-29T18:03:45.184 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQB3SYZjMLnKJRAAQTNcsx5zYcJl4vdyrJ2Oww== 2022-11-29T18:03:45.184 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.2] 2022-11-29T18:03:45.185 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQB6SYZjj/d5KhAAWJNxD6rW2pRTioqAkWcBsg== 2022-11-29T18:03:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.3] 2022-11-29T18:03:45.185 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQB9SYZjy7pbLBAAAi/cqyJyeosulwMzY6R+ew== 2022-11-29T18:03:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2022-11-29T18:03:45.186 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBvSYZj1bS8HhAA7dfrTRaGOeNG5riROdPYsQ== 2022-11-29T18:03:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:[client.1] 2022-11-29T18:03:45.186 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBvSYZj86C1IRAASCCV5uafbadEwOwUEEOGmA== 2022-11-29T18:03:45.187 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-29T18:03:45.228 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-29T18:03:45.273 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-29T18:03:45.276 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-29T18:03:45.324 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-29T18:03:45.327 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-29T18:03:45.372 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-29T18:03:45.415 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-29T18:03:45.461 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-29T18:03:45.464 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-29T18:03:45.508 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.z --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-29T18:03:45.551 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.z --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-29T18:03:45.596 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-29T18:03:45.600 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-29T18:03:45.645 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-29T18:03:45.648 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-29T18:03:45.732 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-29T18:03:45.737 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-29T18:03:45.781 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-29T18:03:45.784 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-29T18:03:45.825 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-29T18:03:45.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-29T18:03:45.917 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-29T18:03:45.920 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-29T18:03:45.977 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-29T18:03:45.978 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-29T18:03:46.022 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-29T18:03:46.075 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-29T18:03:46.090 DEBUG:teuthology.orchestra.run.smithi035:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-29T18:03:46.148 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-29T18:03:46.222 DEBUG:teuthology.orchestra.run.smithi035:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-29T18:03:46.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-29T18:03:46.254 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-29T18:03:46.382 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-29T18:03:46.397 DEBUG:teuthology.orchestra.run.smithi035:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:46.399 DEBUG:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-29T18:03:46.406 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-29T18:03:46.407 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-29T18:03:46.408 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-29T18:03:46.410 INFO:tasks.ceph.mon.a:Started 2022-11-29T18:03:46.411 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-29T18:03:46.411 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-29T18:03:46.414 INFO:tasks.ceph.mon.c:Started 2022-11-29T18:03:46.415 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-29T18:03:46.415 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-29T18:03:46.450 INFO:tasks.ceph.mon.b:Started 2022-11-29T18:03:46.450 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-29T18:03:46.451 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-29T18:03:46.451 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-29T18:03:46.454 INFO:tasks.ceph.mgr.x:Started 2022-11-29T18:03:46.454 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-29T18:03:46.454 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-29T18:03:46.457 INFO:tasks.ceph.mgr.y:Started 2022-11-29T18:03:46.458 INFO:tasks.ceph.mgr.z:Restarting daemon 2022-11-29T18:03:46.458 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i z 2022-11-29T18:03:46.461 INFO:tasks.ceph.mgr.z:Started 2022-11-29T18:03:46.461 INFO:tasks.ceph:Setting crush tunables to default 2022-11-29T18:03:46.461 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd crush tunables default 2022-11-29T18:03:46.567 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d1ddc8700 1 Processor -- start 2022-11-29T18:03:46.568 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d1ddc8700 1 -- start start 2022-11-29T18:03:46.570 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d1ddc8700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:46.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d1ddc8700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d1809e1a0 con 0x7f5d180ad9a0 2022-11-29T18:03:46.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d1ddc8700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d18098ad0 con 0x7f5d180985d0 2022-11-29T18:03:46.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d1ddc8700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d1809ec70 con 0x7f5d180ad5c0 2022-11-29T18:03:46.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d177fe700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:46.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d0ffff700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f5d180ad9a0 0x7f5d1809d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:55092/0 (socket says 172.21.15.35:55092) 2022-11-29T18:03:46.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 learned_addr learned my addr 172.21.15.35:0/1822181783 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:03:46.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-29T18:03:46.574 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:46.574 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:46.575 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.566+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-11-29T18:03:46.771 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.770+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:46.772 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.770+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-29T18:03:46.772 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.770+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:46.773 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.770+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:46.773 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:46.770+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-11-29T18:03:47.175 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.174+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:47.175 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.174+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-29T18:03:47.176 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.174+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:47.176 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.174+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:47.177 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.174+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-11-29T18:03:47.978 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.978+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:47.979 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.978+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-29T18:03:47.979 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.978+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:47.980 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.978+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:47.980 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:47.978+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-11-29T18:03:49.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 msgr2=0x7f5d180980e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-29T18:03:49.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d15ffb700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.35:3300/0 conn(0x7f5d180ad5c0 0x7f5d180980e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:49.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 >> v1:172.21.15.38:6789/0 conn(0x7f5d180985d0 legacy=0x7f5d1809da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:49.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 >> v1:172.21.15.35:6789/0 conn(0x7f5d180ad9a0 legacy=0x7f5d1809d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:49.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d15ffb700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:49.574 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d15ffb700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:49.574 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d1809ec70 con 0x7f5cf0006070 2022-11-29T18:03:49.575 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf0008b50 con 0x7f5cf0003600 2022-11-29T18:03:49.575 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:49.576 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.566+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:49.576 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.570+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-29T18:03:49.576 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.570+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:49.577 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.570+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-29T18:03:49.577 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.570+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:49.577 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.570+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:49.577 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.570+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:49.578 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.570+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-11-29T18:03:49.578 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.570+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-11-29T18:03:49.774 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:49.775 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:49.775 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-29T18:03:49.776 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:49.776 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:49.776 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-11-29T18:03:49.777 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-29T18:03:49.777 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:49.778 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:49.778 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:49.774+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-11-29T18:03:50.179 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:50.179 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:50.180 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-29T18:03:50.180 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:50.181 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:50.181 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-11-29T18:03:50.181 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-29T18:03:50.182 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:50.182 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:50.183 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.178+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2022-11-29T18:03:50.982 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:50.983 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:50.983 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-29T18:03:50.983 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d177fe700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:50.984 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:50.984 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-11-29T18:03:50.984 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-29T18:03:50.984 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d0ffff700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:50.985 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:50.985 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:50.982+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2022-11-29T18:03:52.570 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 msgr2=0x7f5cf0005a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-29T18:03:52.570 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.566+0000 7f5d15ffb700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0003600 0x7f5cf0005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 msgr2=0x7f5cf00084f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-29T18:03:52.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.566+0000 7f5d15ffb700 1 --2- 172.21.15.35:0/1822181783 >> v2:172.21.15.38:3300/0 conn(0x7f5cf0006070 0x7f5cf00084f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.571 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.566+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 reap_dead start 2022-11-29T18:03:52.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.566+0000 7f5d15ffb700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cf00080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:52.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d18097900 con 0x7f5cf0008f70 2022-11-29T18:03:52.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.566+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf0008750 con 0x7f5cf0003a20 2022-11-29T18:03:52.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.570+0000 7f5d15ffb700 1 -- 172.21.15.35:0/1822181783 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1809ec70 con 0x7f5cf0007c00 2022-11-29T18:03:52.572 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.570+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cf00080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:52.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.570+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 msgr2=0x7f5cf00080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-29T18:03:52.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.570+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 msgr2=0x7f5cf00080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:52.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.570+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cf00080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:52.573 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.570+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cf00080b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2022-11-29T18:03:52.774 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.774+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cf00080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:52.775 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.774+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 msgr2=0x7f5cf00080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-29T18:03:52.776 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.774+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 msgr2=0x7f5cf00080b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-29T18:03:52.776 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.774+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cf00080b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2022-11-29T18:03:52.777 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.774+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cf00080b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2022-11-29T18:03:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.798+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747093900 0 0) 0x7f5d18097900 con 0x7f5cf0008f70 2022-11-29T18:03:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.798+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cfc0033f0 con 0x7f5cf0008f70 2022-11-29T18:03:52.801 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.798+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092556290 0 0) 0x7f5cf0008750 con 0x7f5cf0003a20 2022-11-29T18:03:52.802 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.798+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d18097900 con 0x7f5cf0003a20 2022-11-29T18:03:52.805 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.798+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 <== mon.1 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3885222671 0 0) 0x7f5d18097900 con 0x7f5cf0003a20 2022-11-29T18:03:52.805 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.798+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 msgr2=0x7f5cf00080b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-29T18:03:52.805 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.798+0000 7f5d16ffd700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cf00080b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.806 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 >> v1:172.21.15.35:6789/0 conn(0x7f5cf0008f70 legacy=0x7f5cf000b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:52.806 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0010ab0 con 0x7f5cf0003a20 2022-11-29T18:03:52.806 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 <== mon.1 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2443281136 0 0) 0x7f5d00001340 con 0x7f5cf0003a20 2022-11-29T18:03:52.806 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 >> v1:172.21.15.38:6789/0 conn(0x7f5cf0003a20 legacy=0x7f5cf00064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:52.806 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cfc004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:52.807 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cfc004fe0 0x7f5cfc003630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:52.807 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5cfc003b20 0x7f5cfc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:52.807 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1809ec70 con 0x7f5cfc003b20 2022-11-29T18:03:52.807 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc00b9f0 con 0x7f5cfc004fe0 2022-11-29T18:03:52.808 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc00bbb0 con 0x7f5cf0007c00 2022-11-29T18:03:52.808 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d177fe700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cfc004ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:52.808 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cfc004fe0 0x7f5cfc003630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:52.809 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5cfc003b20 0x7f5cfc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:52.809 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 msgr2=0x7f5cfc004ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:52.809 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5cf0007c00 0x7f5cfc004ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.810 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cfc004fe0 msgr2=0x7f5cfc003630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:52.810 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5cfc004fe0 0x7f5cfc003630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.810 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d18097900 con 0x7f5cfc003b20 2022-11-29T18:03:52.811 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5cfc003b20 0x7f5cfc003fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5d180ade10 tx=0x7f5d100027b0).ready entity=mon.0 client_cookie=d5b8606c20a68eee server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:52.811 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10002aa0 con 0x7f5cfc003b20 2022-11-29T18:03:52.811 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1000d040 con 0x7f5cfc003b20 2022-11-29T18:03:52.811 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.802+0000 7f5d16ffd700 1 -- 172.21.15.35:0/1822181783 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10005c40 con 0x7f5cfc003b20 2022-11-29T18:03:52.814 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5cfc003b20 msgr2=0x7f5cfc003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:52.815 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 --2- 172.21.15.35:0/1822181783 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5cfc003b20 0x7f5cfc003fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f5d180ade10 tx=0x7f5d100027b0).stop 2022-11-29T18:03:52.815 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d17fff700 1 -- 172.21.15.35:0/1822181783 reap_dead start 2022-11-29T18:03:52.816 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1822181783 shutdown_connections 2022-11-29T18:03:52.816 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1822181783 >> 172.21.15.35:0/1822181783 conn(0x7f5d180624c0 msgr2=0x7f5d180aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:52.817 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1822181783 shutdown_connections 2022-11-29T18:03:52.817 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1822181783 wait complete. 2022-11-29T18:03:52.817 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 Processor -- start 2022-11-29T18:03:52.818 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- start start 2022-11-29T18:03:52.818 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d1809c600 0x7f5d18093c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:52.818 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d18094f50 0x7f5d18095330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:52.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d1808ff00 0x7f5d18090380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:52.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d18092b60 con 0x7f5d1808ff00 2022-11-29T18:03:52.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d18094600 con 0x7f5d18094f50 2022-11-29T18:03:52.819 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d1ddc8700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d18094880 con 0x7f5d1809c600 2022-11-29T18:03:52.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d177fe700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d1809c600 0x7f5d18093c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:52.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d177fe700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d1809c600 0x7f5d18093c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:36860/0 (socket says 172.21.15.35:36860) 2022-11-29T18:03:52.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d177fe700 1 -- 172.21.15.35:0/1520425275 learned_addr learned my addr 172.21.15.35:0/1520425275 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:03:52.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.806+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d1808ff00 0x7f5d18090380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:52.820 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0ffff700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d18094f50 0x7f5d18095330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:52.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d17fff700 1 -- 172.21.15.35:0/1520425275 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d1809c600 msgr2=0x7f5d18093c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:52.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d1809c600 0x7f5d18093c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d17fff700 1 -- 172.21.15.35:0/1520425275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d18094f50 msgr2=0x7f5d18095330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:52.821 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d18094f50 0x7f5d18095330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d17fff700 1 -- 172.21.15.35:0/1520425275 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d18090870 con 0x7f5d1808ff00 2022-11-29T18:03:52.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d17fff700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d1808ff00 0x7f5d18090380 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f5cfc00b9c0 tx=0x7f5d10017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:52.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 1 -- 172.21.15.35:0/1520425275 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10002aa0 con 0x7f5d1808ff00 2022-11-29T18:03:52.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d18095880 con 0x7f5d1808ff00 2022-11-29T18:03:52.822 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 1 -- 172.21.15.35:0/1520425275 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d10001250 con 0x7f5d1808ff00 2022-11-29T18:03:52.823 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 1 -- 172.21.15.35:0/1520425275 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10013430 con 0x7f5d1808ff00 2022-11-29T18:03:52.823 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d18094160 con 0x7f5d1808ff00 2022-11-29T18:03:52.823 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 1 -- 172.21.15.35:0/1520425275 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f5d10013860 con 0x7f5d1808ff00 2022-11-29T18:03:52.823 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2022-11-29T18:03:52.824 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 4 mgrc handle_mgr_map Got map version 1 2022-11-29T18:03:52.824 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 4 mgrc handle_mgr_map Active mgr is now 2022-11-29T18:03:52.824 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 4 mgrc reconnect No active mgr available yet 2022-11-29T18:03:52.824 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.810+0000 7f5d0f7fe700 1 -- 172.21.15.35:0/1520425275 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f5d100283f0 con 0x7f5d1808ff00 2022-11-29T18:03:52.824 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.814+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d180650b0 con 0x7f5d1808ff00 2022-11-29T18:03:52.825 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.814+0000 7f5d0f7fe700 1 -- 172.21.15.35:0/1520425275 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73156 (secure 0 0 0) 0x7f5d180650b0 con 0x7f5d1808ff00 2022-11-29T18:03:52.883 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.882+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5d1809ca40 con 0x7f5d1808ff00 2022-11-29T18:03:52.889 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d0f7fe700 1 -- 172.21.15.35:0/1520425275 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5d10024070 con 0x7f5d1808ff00 2022-11-29T18:03:52.891 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 10 mgrc shutdown 2022-11-29T18:03:52.891 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d1808ff00 msgr2=0x7f5d18090380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:52.892 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d1808ff00 0x7f5d18090380 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f5cfc00b9c0 tx=0x7f5d10017000).stop 2022-11-29T18:03:52.892 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 shutdown_connections 2022-11-29T18:03:52.892 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d1808ff00 0x7f5d18090380 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.893 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d18094f50 0x7f5d18095330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.893 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 --2- 172.21.15.35:0/1520425275 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d1809c600 0x7f5d18093c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:52.894 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 >> 172.21.15.35:0/1520425275 conn(0x7f5d180624c0 msgr2=0x7f5d18144e10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:52.894 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 shutdown_connections 2022-11-29T18:03:52.895 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:52.890+0000 7f5d1ddc8700 1 -- 172.21.15.35:0/1520425275 wait complete. 2022-11-29T18:03:52.896 INFO:teuthology.orchestra.run.smithi035.stderr:adjusted tunables profile to default 2022-11-29T18:03:52.909 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-29T18:03:52.910 DEBUG:teuthology.orchestra.run.smithi035:> set -ex 2022-11-29T18:03:52.911 DEBUG:teuthology.orchestra.run.smithi035:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-29T18:03:52.926 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:52.927 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-29T18:03:52.944 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:52.945 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-29T18:03:53.003 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2022-11-29T18:03:53.003 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-29T18:03:53.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd new 228fad84-4a8f-44ce-ba6d-f0433a5a22ad 0 2022-11-29T18:03:53.029 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:53.025+0000 7fca37344500 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-29T18:03:53.030 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:53.025+0000 7fa02c861500 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-29T18:03:53.038 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:53.038+0000 7f8fe0015500 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-29T18:03:53.092 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:53.089+0000 7fa02c861500 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-29T18:03:53.093 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:53.089+0000 7fca37344500 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-29T18:03:53.099 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:53.098+0000 7f8fe0015500 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-29T18:03:53.130 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15c96d700 1 Processor -- start 2022-11-29T18:03:53.131 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15c96d700 1 -- start start 2022-11-29T18:03:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15c96d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15809f3c0 0x7fa1581399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15c96d700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa158139ea0 0x7fa15813a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15c96d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15813ae80 con 0x7fa15805d820 2022-11-29T18:03:53.135 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15c96d700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15809ec20 con 0x7fa158139ea0 2022-11-29T18:03:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15c96d700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15813b140 con 0x7fa15809f3c0 2022-11-29T18:03:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15759e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa158139ea0 0x7fa15813a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15759e700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa158139ea0 0x7fa15813a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51816/0 (socket says 172.21.15.38:51816) 2022-11-29T18:03:53.136 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa15759e700 1 -- 172.21.15.38:0/3927391280 learned_addr learned my addr 172.21.15.38:0/3927391280 (peer_addr_for_me v2:172.21.15.38:0/0) 2022-11-29T18:03:53.137 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.125+0000 7fa14ffff700 1 --2- 172.21.15.38:0/3927391280 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15809f3c0 0x7fa1581399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.137 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15759e700 1 -- 172.21.15.38:0/3927391280 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15809f3c0 msgr2=0x7fa1581399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:53.137 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15759e700 1 --2- 172.21.15.38:0/3927391280 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15809f3c0 0x7fa1581399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.137 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15759e700 1 -- 172.21.15.38:0/3927391280 >> v1:172.21.15.38:6789/0 conn(0x7fa15805d820 legacy=0x7fa15809e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.137 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15759e700 1 -- 172.21.15.38:0/3927391280 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15805dc00 con 0x7fa158139ea0 2022-11-29T18:03:53.138 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15659c700 1 -- 172.21.15.38:0/3927391280 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284744424 0 0) 0x7fa15813ae80 con 0x7fa15805d820 2022-11-29T18:03:53.138 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15759e700 1 --2- 172.21.15.38:0/3927391280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa158139ea0 0x7fa15813a330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa150009f20 tx=0x7fa150009840).ready entity=mon.1 client_cookie=3b56dd939a7f88f3 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:53.138 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15659c700 1 -- 172.21.15.38:0/3927391280 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15000e070 con 0x7fa158139ea0 2022-11-29T18:03:53.138 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15659c700 1 -- 172.21.15.38:0/3927391280 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15000bb10 con 0x7fa158139ea0 2022-11-29T18:03:53.139 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15659c700 1 -- 172.21.15.38:0/3927391280 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15000be90 con 0x7fa158139ea0 2022-11-29T18:03:53.139 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 -- 172.21.15.38:0/3927391280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa158139ea0 msgr2=0x7fa15813a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.139 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 --2- 172.21.15.38:0/3927391280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa158139ea0 0x7fa15813a330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fa150009f20 tx=0x7fa150009840).stop 2022-11-29T18:03:53.139 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 -- 172.21.15.38:0/3927391280 shutdown_connections 2022-11-29T18:03:53.139 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 --2- 172.21.15.38:0/3927391280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa158139ea0 0x7fa15813a330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.140 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 --2- 172.21.15.38:0/3927391280 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15809f3c0 0x7fa1581399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.140 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 -- 172.21.15.38:0/3927391280 >> 172.21.15.38:0/3927391280 conn(0x7fa1580624b0 msgr2=0x7fa158063940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:53.140 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 -- 172.21.15.38:0/3927391280 shutdown_connections 2022-11-29T18:03:53.140 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 -- 172.21.15.38:0/3927391280 wait complete. 2022-11-29T18:03:53.141 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 Processor -- start 2022-11-29T18:03:53.141 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 -- start start 2022-11-29T18:03:53.144 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.129+0000 7fa15c96d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15805d820 0x7fa1580acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.144 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15c96d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15809f3c0 0x7fa158067ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.144 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15c96d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa158139ea0 0x7fa1580968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.144 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15c96d700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580684e0 con 0x7fa158139ea0 2022-11-29T18:03:53.145 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15c96d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158097540 con 0x7fa15809f3c0 2022-11-29T18:03:53.145 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15c96d700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580ada60 con 0x7fa15805d820 2022-11-29T18:03:53.145 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15809f3c0 0x7fa158067ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.145 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15809f3c0 0x7fa158067ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51830/0 (socket says 172.21.15.38:51830) 2022-11-29T18:03:53.146 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 -- 172.21.15.38:0/3147183201 learned_addr learned my addr 172.21.15.38:0/3147183201 (peer_addr_for_me v2:172.21.15.38:0/0) 2022-11-29T18:03:53.146 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa156d9d700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15805d820 0x7fa1580acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.146 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15759e700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa158139ea0 0x7fa1580968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.146 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 -- 172.21.15.38:0/3147183201 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15805d820 msgr2=0x7fa1580acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.147 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15805d820 0x7fa1580acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.147 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 -- 172.21.15.38:0/3147183201 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa158139ea0 msgr2=0x7fa1580968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.147 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa158139ea0 0x7fa1580968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.147 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 -- 172.21.15.38:0/3147183201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa150009ae0 con 0x7fa15809f3c0 2022-11-29T18:03:53.148 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14ffff700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15809f3c0 0x7fa158067ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fa14000d890 tx=0x7fa140008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:53.148 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 1 -- 172.21.15.38:0/3147183201 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1400125f0 con 0x7fa15809f3c0 2022-11-29T18:03:53.148 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 1 -- 172.21.15.38:0/3147183201 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa140012750 con 0x7fa15809f3c0 2022-11-29T18:03:53.148 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1580ad2e0 con 0x7fa15809f3c0 2022-11-29T18:03:53.148 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 1 -- 172.21.15.38:0/3147183201 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140012ad0 con 0x7fa15809f3c0 2022-11-29T18:03:53.149 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa158068330 con 0x7fa15809f3c0 2022-11-29T18:03:53.149 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 1 -- 172.21.15.38:0/3147183201 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fa140005ae0 con 0x7fa15809f3c0 2022-11-29T18:03:53.149 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2022-11-29T18:03:53.149 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 4 mgrc handle_mgr_map Got map version 1 2022-11-29T18:03:53.150 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 4 mgrc handle_mgr_map Active mgr is now 2022-11-29T18:03:53.150 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 4 mgrc reconnect No active mgr available yet 2022-11-29T18:03:53.150 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa14effd700 1 -- 172.21.15.38:0/3147183201 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7fa140006310 con 0x7fa15809f3c0 2022-11-29T18:03:53.150 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.133+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa158068650 con 0x7fa15809f3c0 2022-11-29T18:03:53.150 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.137+0000 7fa14effd700 1 -- 172.21.15.38:0/3147183201 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73156 (secure 0 0 0) 0x7fa1400065a0 con 0x7fa15809f3c0 2022-11-29T18:03:53.215 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.209+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd new", "uuid": "228fad84-4a8f-44ce-ba6d-f0433a5a22ad", "id": 0} v 0) v1 -- 0x7fa158046830 con 0x7fa15809f3c0 2022-11-29T18:03:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa14effd700 1 -- 172.21.15.38:0/3147183201 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "228fad84-4a8f-44ce-ba6d-f0433a5a22ad", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa140006740 con 0x7fa15809f3c0 2022-11-29T18:03:53.225 INFO:teuthology.orchestra.run.smithi038.stdout:0 2022-11-29T18:03:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 10 mgrc shutdown 2022-11-29T18:03:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15809f3c0 msgr2=0x7fa158067ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15809f3c0 0x7fa158067ca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fa14000d890 tx=0x7fa140008040).stop 2022-11-29T18:03:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 shutdown_connections 2022-11-29T18:03:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa158139ea0 0x7fa1580968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa15809f3c0 0x7fa158067ca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 --2- 172.21.15.38:0/3147183201 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa15805d820 0x7fa1580acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 >> 172.21.15.38:0/3147183201 conn(0x7fa1580624b0 msgr2=0x7fa15813b490 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 shutdown_connections 2022-11-29T18:03:53.229 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.221+0000 7fa15c96d700 1 -- 172.21.15.38:0/3147183201 wait complete. 2022-11-29T18:03:53.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd new d58467d0-703a-4da6-980d-30bf1051838b 1 2022-11-29T18:03:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.337+0000 7f7e64aa8700 1 Processor -- start 2022-11-29T18:03:53.353 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.337+0000 7f7e64aa8700 1 -- start start 2022-11-29T18:03:53.354 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e600983b0 0x7f7e6012e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6012ec80 con 0x7f7e600650d0 2022-11-29T18:03:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6013b900 con 0x7f7e6012e550 2022-11-29T18:03:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6012e9c0 con 0x7f7e600983b0 2022-11-29T18:03:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5effd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e600983b0 0x7f7e6012e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5effd700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e600983b0 0x7f7e6012e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51838/0 (socket says 172.21.15.38:51838) 2022-11-29T18:03:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5ffff700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7e6012e550 0x7f7e601429d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51110/0 (socket says 172.21.15.38:51110) 2022-11-29T18:03:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5effd700 1 -- 172.21.15.38:0/1506165711 learned_addr learned my addr 172.21.15.38:0/1506165711 (peer_addr_for_me v2:172.21.15.38:0/0) 2022-11-29T18:03:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5effd700 1 -- 172.21.15.38:0/1506165711 >> v1:172.21.15.38:6789/0 conn(0x7f7e6012e550 legacy=0x7f7e601429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5effd700 1 -- 172.21.15.38:0/1506165711 >> v1:172.21.15.35:6789/0 conn(0x7f7e600650d0 legacy=0x7f7e6013bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.357 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5effd700 1 -- 172.21.15.38:0/1506165711 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6009ef20 con 0x7f7e600983b0 2022-11-29T18:03:53.357 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5effd700 1 --2- 172.21.15.38:0/1506165711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600983b0 0x7f7e6012e060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f7e48006fd0 tx=0x7f7e480090f0).ready entity=mon.1 client_cookie=c0ea8212c45c6738 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:53.357 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5e7fc700 1 -- 172.21.15.38:0/1506165711 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800b600 con 0x7f7e600983b0 2022-11-29T18:03:53.357 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5e7fc700 1 -- 172.21.15.38:0/1506165711 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4800b760 con 0x7f7e600983b0 2022-11-29T18:03:53.358 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e5e7fc700 1 -- 172.21.15.38:0/1506165711 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800bae0 con 0x7f7e600983b0 2022-11-29T18:03:53.358 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- 172.21.15.38:0/1506165711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600983b0 msgr2=0x7f7e6012e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.358 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 --2- 172.21.15.38:0/1506165711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600983b0 0x7f7e6012e060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f7e48006fd0 tx=0x7f7e480090f0).stop 2022-11-29T18:03:53.358 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- 172.21.15.38:0/1506165711 shutdown_connections 2022-11-29T18:03:53.359 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 --2- 172.21.15.38:0/1506165711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600983b0 0x7f7e6012e060 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.359 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- 172.21.15.38:0/1506165711 >> 172.21.15.38:0/1506165711 conn(0x7f7e600619f0 msgr2=0x7f7e60061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:53.360 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- 172.21.15.38:0/1506165711 shutdown_connections 2022-11-29T18:03:53.360 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- 172.21.15.38:0/1506165711 wait complete. 2022-11-29T18:03:53.360 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 Processor -- start 2022-11-29T18:03:53.360 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.341+0000 7f7e64aa8700 1 -- start start 2022-11-29T18:03:53.361 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600650d0 0x7f7e60093900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.361 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7e600983b0 0x7f7e601389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.361 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f7e6012e550 0x7f7e600925f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.361 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60148af0 con 0x7f7e600983b0 2022-11-29T18:03:53.361 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60148db0 con 0x7f7e600650d0 2022-11-29T18:03:53.362 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60093df0 con 0x7f7e6012e550 2022-11-29T18:03:53.362 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600650d0 0x7f7e60093900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.362 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600650d0 0x7f7e60093900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51850/0 (socket says 172.21.15.38:51850) 2022-11-29T18:03:53.362 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 -- 172.21.15.38:0/2158621505 learned_addr learned my addr 172.21.15.38:0/2158621505 (peer_addr_for_me v2:172.21.15.38:0/0) 2022-11-29T18:03:53.363 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5effd700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7e600983b0 0x7f7e601389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.363 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5ffff700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f7e6012e550 0x7f7e600925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.363 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 -- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f7e6012e550 msgr2=0x7f7e600925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.363 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f7e6012e550 0x7f7e600925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.364 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 -- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7e600983b0 msgr2=0x7f7e601389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.364 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7e600983b0 0x7f7e601389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.364 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 -- 172.21.15.38:0/2158621505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48009370 con 0x7f7e600650d0 2022-11-29T18:03:53.364 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5effd700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7e600983b0 0x7f7e601389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-29T18:03:53.364 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5ffff700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f7e6012e550 0x7f7e600925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-29T18:03:53.365 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e5f7fe700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600650d0 0x7f7e60093900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f7e5000c810 tx=0x7f7e50001640).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:53.365 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 1 -- 172.21.15.38:0/2158621505 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000a580 con 0x7f7e600650d0 2022-11-29T18:03:53.365 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 1 -- 172.21.15.38:0/2158621505 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000a6e0 con 0x7f7e600650d0 2022-11-29T18:03:53.365 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e60138e90 con 0x7f7e600650d0 2022-11-29T18:03:53.366 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 1 -- 172.21.15.38:0/2158621505 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000aa80 con 0x7f7e600650d0 2022-11-29T18:03:53.366 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6009e210 con 0x7f7e600650d0 2022-11-29T18:03:53.366 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 1 -- 172.21.15.38:0/2158621505 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f7e50019030 con 0x7f7e600650d0 2022-11-29T18:03:53.366 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2022-11-29T18:03:53.366 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 4 mgrc handle_mgr_map Got map version 1 2022-11-29T18:03:53.367 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 4 mgrc handle_mgr_map Active mgr is now 2022-11-29T18:03:53.367 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 4 mgrc reconnect No active mgr available yet 2022-11-29T18:03:53.367 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e577fe700 1 -- 172.21.15.38:0/2158621505 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f7e50012400 con 0x7f7e600650d0 2022-11-29T18:03:53.367 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.345+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e60046830 con 0x7f7e600650d0 2022-11-29T18:03:53.368 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.349+0000 7f7e577fe700 1 -- 172.21.15.38:0/2158621505 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73156 (secure 0 0 0) 0x7f7e50021060 con 0x7f7e600650d0 2022-11-29T18:03:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.425+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d58467d0-703a-4da6-980d-30bf1051838b", "id": 1} v 0) v1 -- 0x7f7e6009f1e0 con 0x7f7e600650d0 2022-11-29T18:03:53.443 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.437+0000 7f7e577fe700 1 -- 172.21.15.38:0/2158621505 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d58467d0-703a-4da6-980d-30bf1051838b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7e5002a0b0 con 0x7f7e600650d0 2022-11-29T18:03:53.443 INFO:teuthology.orchestra.run.smithi038.stdout:1 2022-11-29T18:03:53.444 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.437+0000 7f7e64aa8700 10 mgrc shutdown 2022-11-29T18:03:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600650d0 msgr2=0x7f7e60093900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600650d0 0x7f7e60093900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f7e5000c810 tx=0x7f7e50001640).stop 2022-11-29T18:03:53.445 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 shutdown_connections 2022-11-29T18:03:53.446 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f7e6012e550 0x7f7e600925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.446 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f7e600983b0 0x7f7e601389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.446 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 --2- 172.21.15.38:0/2158621505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e600650d0 0x7f7e60093900 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.446 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 >> 172.21.15.38:0/2158621505 conn(0x7f7e600619f0 msgr2=0x7f7e60068c00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:53.447 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 shutdown_connections 2022-11-29T18:03:53.447 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.441+0000 7f7e64aa8700 1 -- 172.21.15.38:0/2158621505 wait complete. 2022-11-29T18:03:53.459 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd new 34a91d7b-b17b-4400-b2f3-0de86be92ac3 2 2022-11-29T18:03:53.561 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc930fda700 1 Processor -- start 2022-11-29T18:03:53.561 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc930fda700 1 -- start start 2022-11-29T18:03:53.565 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc930fda700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fc92c09f3c0 0x7fc92c1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.565 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc930fda700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc92c139ea0 0x7fc92c13a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.565 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc930fda700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c13ae80 con 0x7fc92c05d820 2022-11-29T18:03:53.566 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc930fda700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c09ec20 con 0x7fc92c09f3c0 2022-11-29T18:03:53.566 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc930fda700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c13b140 con 0x7fc92c139ea0 2022-11-29T18:03:53.567 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92bfff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc92c139ea0 0x7fc92c13a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.567 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92b7fe700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc92c05d820 0x7fc92c09e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51126/0 (socket says 172.21.15.38:51126) 2022-11-29T18:03:53.568 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92b7fe700 1 -- 172.21.15.38:0/4212187261 learned_addr learned my addr 172.21.15.38:0/4212187261 (peer_addr_for_me v1:172.21.15.38:0/0) 2022-11-29T18:03:53.568 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92affd700 1 --2- 172.21.15.38:0/4212187261 >> v2:172.21.15.35:3300/0 conn(0x7fc92c09f3c0 0x7fc92c1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.569 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92bfff700 1 -- 172.21.15.38:0/4212187261 >> v2:172.21.15.35:3300/0 conn(0x7fc92c09f3c0 msgr2=0x7fc92c1399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:53.569 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92bfff700 1 --2- 172.21.15.38:0/4212187261 >> v2:172.21.15.35:3300/0 conn(0x7fc92c09f3c0 0x7fc92c1399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.570 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92bfff700 1 -- 172.21.15.38:0/4212187261 >> v1:172.21.15.38:6789/0 conn(0x7fc92c05d820 legacy=0x7fc92c09e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.570 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92a7fc700 1 -- 172.21.15.38:0/4212187261 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649835138 0 0) 0x7fc92c13ae80 con 0x7fc92c05d820 2022-11-29T18:03:53.571 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92bfff700 1 -- 172.21.15.38:0/4212187261 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c05dc00 con 0x7fc92c139ea0 2022-11-29T18:03:53.571 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92bfff700 1 --2- 172.21.15.38:0/4212187261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c13a2d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc92c0a0150 tx=0x7fc924009780).ready entity=mon.1 client_cookie=3b3ead54b2d159a4 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:53.572 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92a7fc700 1 -- 172.21.15.38:0/4212187261 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92400e070 con 0x7fc92c139ea0 2022-11-29T18:03:53.572 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92a7fc700 1 -- 172.21.15.38:0/4212187261 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92400bcc0 con 0x7fc92c139ea0 2022-11-29T18:03:53.572 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.557+0000 7fc92a7fc700 1 -- 172.21.15.38:0/4212187261 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924012620 con 0x7fc92c139ea0 2022-11-29T18:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- 172.21.15.38:0/4212187261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 msgr2=0x7fc92c13a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.573 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 --2- 172.21.15.38:0/4212187261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c13a2d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc92c0a0150 tx=0x7fc924009780).stop 2022-11-29T18:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- 172.21.15.38:0/4212187261 shutdown_connections 2022-11-29T18:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 --2- 172.21.15.38:0/4212187261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c13a2d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.574 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 --2- 172.21.15.38:0/4212187261 >> v2:172.21.15.35:3300/0 conn(0x7fc92c09f3c0 0x7fc92c1399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.575 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- 172.21.15.38:0/4212187261 >> 172.21.15.38:0/4212187261 conn(0x7fc92c0624b0 msgr2=0x7fc92c063940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:53.575 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- 172.21.15.38:0/4212187261 shutdown_connections 2022-11-29T18:03:53.576 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- 172.21.15.38:0/4212187261 wait complete. 2022-11-29T18:03:53.581 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 Processor -- start 2022-11-29T18:03:53.582 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- start start 2022-11-29T18:03:53.582 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc92c05d820 0x7fc92c09c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.582 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fc92c09f3c0 0x7fc92c14d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.582 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c0967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.582 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0677e0 con 0x7fc92c05d820 2022-11-29T18:03:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c14ddf0 con 0x7fc92c139ea0 2022-11-29T18:03:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc930fda700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c14e0b0 con 0x7fc92c09f3c0 2022-11-29T18:03:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c0967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc92bfff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c0967f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51868/0 (socket says 172.21.15.38:51868) 2022-11-29T18:03:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc92bfff700 1 -- 172.21.15.38:0/2490307261 learned_addr learned my addr 172.21.15.38:0/2490307261 (peer_addr_for_me v2:172.21.15.38:0/0) 2022-11-29T18:03:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc92b7fe700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc92c05d820 0x7fc92c09c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.561+0000 7fc92affd700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fc92c09f3c0 0x7fc92c14d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc92bfff700 1 -- 172.21.15.38:0/2490307261 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fc92c09f3c0 msgr2=0x7fc92c14d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc92bfff700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fc92c09f3c0 0x7fc92c14d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc92bfff700 1 -- 172.21.15.38:0/2490307261 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc92c05d820 msgr2=0x7fc92c09c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.586 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc92bfff700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc92c05d820 0x7fc92c09c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.586 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc92bfff700 1 -- 172.21.15.38:0/2490307261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924009a20 con 0x7fc92c139ea0 2022-11-29T18:03:53.586 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc92bfff700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c0967f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fc92c09e470 tx=0x7fc924009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:53.586 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 1 -- 172.21.15.38:0/2490307261 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92401d630 con 0x7fc92c139ea0 2022-11-29T18:03:53.586 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 1 -- 172.21.15.38:0/2490307261 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92401d790 con 0x7fc92c139ea0 2022-11-29T18:03:53.587 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c068d30 con 0x7fc92c139ea0 2022-11-29T18:03:53.587 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 1 -- 172.21.15.38:0/2490307261 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92401db50 con 0x7fc92c139ea0 2022-11-29T18:03:53.587 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c097d10 con 0x7fc92c139ea0 2022-11-29T18:03:53.587 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 1 -- 172.21.15.38:0/2490307261 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fc924026670 con 0x7fc92c139ea0 2022-11-29T18:03:53.588 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2022-11-29T18:03:53.588 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 4 mgrc handle_mgr_map Got map version 1 2022-11-29T18:03:53.588 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 4 mgrc handle_mgr_map Active mgr is now 2022-11-29T18:03:53.588 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 4 mgrc reconnect No active mgr available yet 2022-11-29T18:03:53.588 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc9137fe700 1 -- 172.21.15.38:0/2490307261 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fc9240087f0 con 0x7fc92c139ea0 2022-11-29T18:03:53.589 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.565+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c046830 con 0x7fc92c139ea0 2022-11-29T18:03:53.589 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.569+0000 7fc9137fe700 1 -- 172.21.15.38:0/2490307261 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73156 (secure 0 0 0) 0x7fc924017070 con 0x7fc92c139ea0 2022-11-29T18:03:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.645+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd new", "uuid": "34a91d7b-b17b-4400-b2f3-0de86be92ac3", "id": 2} v 0) v1 -- 0x7fc92c13a7c0 con 0x7fc92c139ea0 2022-11-29T18:03:53.657 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc9137fe700 1 -- 172.21.15.38:0/2490307261 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "34a91d7b-b17b-4400-b2f3-0de86be92ac3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc924026a40 con 0x7fc92c139ea0 2022-11-29T18:03:53.657 INFO:teuthology.orchestra.run.smithi038.stdout:2 2022-11-29T18:03:53.659 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 10 mgrc shutdown 2022-11-29T18:03:53.660 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 msgr2=0x7fc92c0967f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.660 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c0967f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fc92c09e470 tx=0x7fc924009780).stop 2022-11-29T18:03:53.660 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 shutdown_connections 2022-11-29T18:03:53.660 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc92c139ea0 0x7fc92c0967f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.661 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fc92c09f3c0 0x7fc92c14d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.661 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 --2- 172.21.15.38:0/2490307261 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fc92c05d820 0x7fc92c09c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.661 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 >> 172.21.15.38:0/2490307261 conn(0x7fc92c0624b0 msgr2=0x7fc92c063940 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:53.661 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 shutdown_connections 2022-11-29T18:03:53.661 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.653+0000 7fc930fda700 1 -- 172.21.15.38:0/2490307261 wait complete. 2022-11-29T18:03:53.679 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd new c7d3a292-4f7e-456d-ad04-0a67f33eb539 3 2022-11-29T18:03:53.749 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:53.745+0000 7fa02c861500 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-29T18:03:53.750 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:53.745+0000 7fca37344500 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-29T18:03:53.779 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.773+0000 7f14469bd700 1 Processor -- start 2022-11-29T18:03:53.779 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.773+0000 7f14469bd700 1 -- start start 2022-11-29T18:03:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f144009f330 0x7f144009f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1440068160 con 0x7f14400652d0 2022-11-29T18:03:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14400687e0 con 0x7f144013d940 2022-11-29T18:03:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144013e5d0 con 0x7f144009f330 2022-11-29T18:03:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14461bc700 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f144013d940 0x7f144013ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51130/0 (socket says 172.21.15.38:51130) 2022-11-29T18:03:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14461bc700 1 -- 172.21.15.38:0/526599064 learned_addr learned my addr 172.21.15.38:0/526599064 (peer_addr_for_me v1:172.21.15.38:0/0) 2022-11-29T18:03:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14451ba700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f144009f330 0x7f144009f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278929363 0 0) 0x7f14400687e0 con 0x7f144013d940 2022-11-29T18:03:53.792 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14280033f0 con 0x7f144013d940 2022-11-29T18:03:53.792 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1470077636 0 0) 0x7f1440068160 con 0x7f14400652d0 2022-11-29T18:03:53.792 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14400687e0 con 0x7f14400652d0 2022-11-29T18:03:53.792 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 <== mon.1 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2359641176 0 0) 0x7f14280033f0 con 0x7f144013d940 2022-11-29T18:03:53.793 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f144009f330 msgr2=0x7f144009f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:53.793 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f144009f330 0x7f144009f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.793 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 >> v1:172.21.15.35:6789/0 conn(0x7f14400652d0 legacy=0x7f1440067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.793 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400656b0 con 0x7f144013d940 2022-11-29T18:03:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 <== mon.1 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2443281136 0 0) 0x7f14380017f0 con 0x7f144013d940 2022-11-29T18:03:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 >> v1:172.21.15.38:6789/0 conn(0x7f144013d940 legacy=0x7f144013ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1428003aa0 0x7f1428003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f1428004690 0x7f1428004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f144009f330 0x7f1428005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144013e5d0 con 0x7f1428004690 2022-11-29T18:03:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142800b990 con 0x7f1428003aa0 2022-11-29T18:03:53.797 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142800bb50 con 0x7f144009f330 2022-11-29T18:03:53.797 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14459bb700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1428003aa0 0x7f1428003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.798 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14451ba700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f1428004690 0x7f1428004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.798 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14461bc700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f144009f330 0x7f1428005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.799 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14459bb700 1 -- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f144009f330 msgr2=0x7f1428005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:53.799 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14459bb700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f144009f330 0x7f1428005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.799 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14459bb700 1 -- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f1428004690 msgr2=0x7f1428004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:53.800 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14459bb700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f1428004690 0x7f1428004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.800 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14459bb700 1 -- 172.21.15.38:0/526599064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280033f0 con 0x7f1428003aa0 2022-11-29T18:03:53.800 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14459bb700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1428003aa0 0x7f1428003ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f143c001890 tx=0x7f143c007bb0).ready entity=mon.1 client_cookie=155f7be80a01e0a3 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:53.800 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c007020 con 0x7f1428003aa0 2022-11-29T18:03:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c007180 con 0x7f1428003aa0 2022-11-29T18:03:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14449b9700 1 -- 172.21.15.38:0/526599064 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c007500 con 0x7f1428003aa0 2022-11-29T18:03:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 -- 172.21.15.38:0/526599064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1428003aa0 msgr2=0x7f1428003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.801 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 --2- 172.21.15.38:0/526599064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1428003aa0 0x7f1428003ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f143c001890 tx=0x7f143c007bb0).stop 2022-11-29T18:03:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14461bc700 1 -- 172.21.15.38:0/526599064 reap_dead start 2022-11-29T18:03:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 -- 172.21.15.38:0/526599064 shutdown_connections 2022-11-29T18:03:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 -- 172.21.15.38:0/526599064 >> 172.21.15.38:0/526599064 conn(0x7f14400ae030 msgr2=0x7f1440062b00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:53.802 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.777+0000 7f14469bd700 1 -- 172.21.15.38:0/526599064 shutdown_connections 2022-11-29T18:03:53.803 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 -- 172.21.15.38:0/526599064 wait complete. 2022-11-29T18:03:53.803 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 Processor -- start 2022-11-29T18:03:53.803 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 -- start start 2022-11-29T18:03:53.804 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f1428003aa0 0x7f1440139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.804 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144009afc0 0x7f144014d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.804 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f144014d710 0x7f1440096480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:53.805 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144009b3e0 con 0x7f144014d710 2022-11-29T18:03:53.805 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144014dba0 con 0x7f144009afc0 2022-11-29T18:03:53.805 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144014de60 con 0x7f1428003aa0 2022-11-29T18:03:53.806 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144009afc0 0x7f144014d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.806 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144009afc0 0x7f144014d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51884/0 (socket says 172.21.15.38:51884) 2022-11-29T18:03:53.807 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 -- 172.21.15.38:0/3957486837 learned_addr learned my addr 172.21.15.38:0/3957486837 (peer_addr_for_me v2:172.21.15.38:0/0) 2022-11-29T18:03:53.807 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14459bb700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f1428003aa0 0x7f1440139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.808 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14461bc700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f144014d710 0x7f1440096480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:53.808 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 -- 172.21.15.38:0/3957486837 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f1428003aa0 msgr2=0x7f1440139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.808 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f1428003aa0 0x7f1440139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.809 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 -- 172.21.15.38:0/3957486837 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f144014d710 msgr2=0x7f1440096480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.809 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f144014d710 0x7f1440096480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.809 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 -- 172.21.15.38:0/3957486837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440061ed0 con 0x7f144009afc0 2022-11-29T18:03:53.809 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14451ba700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144009afc0 0x7f144014d220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f14300119f0 tx=0x7f1430009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:53.810 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f142dffb700 1 -- 172.21.15.38:0/3957486837 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000f820 con 0x7f144009afc0 2022-11-29T18:03:53.810 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f142dffb700 1 -- 172.21.15.38:0/3957486837 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143000f980 con 0x7f144009afc0 2022-11-29T18:03:53.810 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1440139e30 con 0x7f144009afc0 2022-11-29T18:03:53.810 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.781+0000 7f142dffb700 1 -- 172.21.15.38:0/3957486837 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143000fd00 con 0x7f144009afc0 2022-11-29T18:03:53.810 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.785+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14401401d0 con 0x7f144009afc0 2022-11-29T18:03:53.811 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.785+0000 7f142dffb700 1 -- 172.21.15.38:0/3957486837 <== mon.1 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7f1430024070 con 0x7f144009afc0 2022-11-29T18:03:53.811 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.785+0000 7f142dffb700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2022-11-29T18:03:53.812 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.785+0000 7f142dffb700 4 mgrc handle_mgr_map Got map version 1 2022-11-29T18:03:53.812 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.785+0000 7f142dffb700 4 mgrc handle_mgr_map Active mgr is now 2022-11-29T18:03:53.812 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.785+0000 7f142dffb700 4 mgrc reconnect No active mgr available yet 2022-11-29T18:03:53.812 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.785+0000 7f142dffb700 1 -- 172.21.15.38:0/3957486837 <== mon.1 v2:172.21.15.38:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f143000fe60 con 0x7f144009afc0 2022-11-29T18:03:53.812 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.785+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144009d660 con 0x7f144009afc0 2022-11-29T18:03:53.813 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.789+0000 7f142dffb700 1 -- 172.21.15.38:0/3957486837 <== mon.1 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73156 (secure 0 0 0) 0x7f143003cde0 con 0x7f144009afc0 2022-11-29T18:03:53.814 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:53.798+0000 7f8fe0015500 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-29T18:03:53.871 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.865+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c7d3a292-4f7e-456d-ad04-0a67f33eb539", "id": 3} v 0) v1 -- 0x7f144009b590 con 0x7f144009afc0 2022-11-29T18:03:53.878 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f142dffb700 1 -- 172.21.15.38:0/3957486837 <== mon.1 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c7d3a292-4f7e-456d-ad04-0a67f33eb539", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f143002b030 con 0x7f144009afc0 2022-11-29T18:03:53.879 INFO:teuthology.orchestra.run.smithi038.stdout:3 2022-11-29T18:03:53.881 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 10 mgrc shutdown 2022-11-29T18:03:53.882 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144009afc0 msgr2=0x7f144014d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:53.882 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144009afc0 0x7f144014d220 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f14300119f0 tx=0x7f1430009060).stop 2022-11-29T18:03:53.883 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 shutdown_connections 2022-11-29T18:03:53.883 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f144014d710 0x7f1440096480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.884 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f144009afc0 0x7f144014d220 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.884 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 --2- 172.21.15.38:0/3957486837 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f1428003aa0 0x7f1440139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:53.884 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 >> 172.21.15.38:0/3957486837 conn(0x7f14400ae030 msgr2=0x7f14400ac810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:53.885 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 shutdown_connections 2022-11-29T18:03:53.885 INFO:teuthology.orchestra.run.smithi038.stderr:2022-11-29T18:03:53.873+0000 7f14469bd700 1 -- 172.21.15.38:0/3957486837 wait complete. 2022-11-29T18:03:53.893 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-29T18:03:53.894 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-29T18:03:53.897 INFO:tasks.ceph.osd.0:Started 2022-11-29T18:03:53.898 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-29T18:03:53.898 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-29T18:03:53.901 INFO:tasks.ceph.osd.1:Started 2022-11-29T18:03:53.901 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-29T18:03:53.901 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-29T18:03:53.904 INFO:tasks.ceph.osd.2:Started 2022-11-29T18:03:53.905 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-29T18:03:53.906 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-29T18:03:53.908 INFO:tasks.ceph.osd.3:Started 2022-11-29T18:03:53.909 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-29T18:03:54.019 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 Processor -- start 2022-11-29T18:03:54.019 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- start start 2022-11-29T18:03:54.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fe6f405d820 0x7fe6f4095190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe6f4093de0 0x7fe6f4130680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f40944c0 con 0x7fe6f4130b70 2022-11-29T18:03:54.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f409f6d0 con 0x7fe6f405d820 2022-11-29T18:03:54.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f4095680 con 0x7fe6f4093de0 2022-11-29T18:03:54.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f2d9d700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7fe6f405d820 0x7fe6f4095190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f359e700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fe6f4130b70 0x7fe6f413bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50746/0 (socket says 172.21.15.35:50746) 2022-11-29T18:03:54.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f359e700 1 -- 172.21.15.35:0/1788023827 learned_addr learned my addr 172.21.15.35:0/1788023827 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:03:54.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6ebfff700 1 --2- 172.21.15.35:0/1788023827 >> v2:172.21.15.38:3300/0 conn(0x7fe6f4093de0 0x7fe6f4130680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f2d9d700 1 -- 172.21.15.35:0/1788023827 >> v2:172.21.15.38:3300/0 conn(0x7fe6f4093de0 msgr2=0x7fe6f4130680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:54.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f2d9d700 1 --2- 172.21.15.35:0/1788023827 >> v2:172.21.15.38:3300/0 conn(0x7fe6f4093de0 0x7fe6f4130680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f2d9d700 1 -- 172.21.15.35:0/1788023827 >> v1:172.21.15.35:6789/0 conn(0x7fe6f4130b70 legacy=0x7fe6f413bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f259c700 1 -- 172.21.15.35:0/1788023827 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408656100 0 0) 0x7fe6f40944c0 con 0x7fe6f4130b70 2022-11-29T18:03:54.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f2d9d700 1 -- 172.21.15.35:0/1788023827 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4130fa0 con 0x7fe6f405d820 2022-11-29T18:03:54.026 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f2d9d700 1 --2- 172.21.15.35:0/1788023827 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f405d820 0x7fe6f4095190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fe6f4095be0 tx=0x7fe6e4009060).ready entity=mon.0 client_cookie=95e555ceaf362023 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:54.026 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f259c700 1 -- 172.21.15.35:0/1788023827 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400f510 con 0x7fe6f405d820 2022-11-29T18:03:54.026 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f259c700 1 -- 172.21.15.35:0/1788023827 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400f670 con 0x7fe6f405d820 2022-11-29T18:03:54.026 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f259c700 1 -- 172.21.15.35:0/1788023827 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400f9f0 con 0x7fe6f405d820 2022-11-29T18:03:54.027 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- 172.21.15.35:0/1788023827 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f405d820 msgr2=0x7fe6f4095190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.027 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 --2- 172.21.15.35:0/1788023827 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f405d820 0x7fe6f4095190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fe6f4095be0 tx=0x7fe6e4009060).stop 2022-11-29T18:03:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- 172.21.15.35:0/1788023827 shutdown_connections 2022-11-29T18:03:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 --2- 172.21.15.35:0/1788023827 >> v2:172.21.15.38:3300/0 conn(0x7fe6f4093de0 0x7fe6f4130680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 --2- 172.21.15.35:0/1788023827 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f405d820 0x7fe6f4095190 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- 172.21.15.35:0/1788023827 >> 172.21.15.35:0/1788023827 conn(0x7fe6f40624c0 msgr2=0x7fe6f40acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- 172.21.15.35:0/1788023827 shutdown_connections 2022-11-29T18:03:54.029 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.018+0000 7fe6f8851700 1 -- 172.21.15.35:0/1788023827 wait complete. 2022-11-29T18:03:54.033 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 Processor -- start 2022-11-29T18:03:54.033 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 -- start start 2022-11-29T18:03:54.033 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fe6f405d820 0x7fe6f40a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6f4093de0 0x7fe6f40987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f4130b70 0x7fe6f40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40a1e70 con 0x7fe6f4130b70 2022-11-29T18:03:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f412f570 con 0x7fe6f4093de0 2022-11-29T18:03:54.034 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f414e090 con 0x7fe6f405d820 2022-11-29T18:03:54.035 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fe6f405d820 0x7fe6f40a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.035 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fe6f405d820 0x7fe6f40a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:36870/0 (socket says 172.21.15.35:36870) 2022-11-29T18:03:54.035 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 -- 172.21.15.35:0/1247844360 learned_addr learned my addr 172.21.15.35:0/1247844360 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:03:54.035 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f359e700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f4130b70 0x7fe6f40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.036 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6ebfff700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6f4093de0 0x7fe6f40987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.036 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 -- 172.21.15.35:0/1247844360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6f4093de0 msgr2=0x7fe6f40987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.036 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6f4093de0 0x7fe6f40987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.036 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 -- 172.21.15.35:0/1247844360 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f4130b70 msgr2=0x7fe6f40ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.036 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f4130b70 0x7fe6f40ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 -- 172.21.15.35:0/1247844360 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e40092a0 con 0x7fe6f405d820 2022-11-29T18:03:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f2d9d700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fe6f405d820 0x7fe6f40a1470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fe6f40659d0 tx=0x7fe6e4017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6eaffd700 1 -- 172.21.15.35:0/1247844360 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e401a4d0 con 0x7fe6f405d820 2022-11-29T18:03:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6eaffd700 1 -- 172.21.15.35:0/1247844360 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e401e070 con 0x7fe6f405d820 2022-11-29T18:03:54.037 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f414e350 con 0x7fe6f405d820 2022-11-29T18:03:54.038 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6eaffd700 1 -- 172.21.15.35:0/1247844360 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e401a860 con 0x7fe6f405d820 2022-11-29T18:03:54.038 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.022+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f40a1960 con 0x7fe6f405d820 2022-11-29T18:03:54.038 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.026+0000 7fe6eaffd700 1 -- 172.21.15.35:0/1247844360 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7fe6e4029020 con 0x7fe6f405d820 2022-11-29T18:03:54.038 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.026+0000 7fe6eaffd700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2022-11-29T18:03:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.026+0000 7fe6eaffd700 4 mgrc handle_mgr_map Got map version 1 2022-11-29T18:03:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.026+0000 7fe6eaffd700 4 mgrc handle_mgr_map Active mgr is now 2022-11-29T18:03:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.026+0000 7fe6eaffd700 4 mgrc reconnect No active mgr available yet 2022-11-29T18:03:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.026+0000 7fe6eaffd700 1 -- 172.21.15.35:0/1247844360 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7fe6e402cc90 con 0x7fe6f405d820 2022-11-29T18:03:54.039 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.026+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f4046830 con 0x7fe6f405d820 2022-11-29T18:03:54.040 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.030+0000 7fe6eaffd700 1 -- 172.21.15.35:0/1247844360 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73156 (secure 0 0 0) 0x7fe6e402c760 con 0x7fe6f405d820 2022-11-29T18:03:54.093 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6f40ac190 con 0x7fe6f405d820 2022-11-29T18:03:54.094 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6eaffd700 1 -- 172.21.15.35:0/1247844360 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4140 (secure 0 0 0) 0x7fe6e402c900 con 0x7fe6f405d820 2022-11-29T18:03:54.095 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:03:54.095 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":6,"fsid":"338b2b5b-2e59-4e89-9321-f2b909af04fd","created":"2022-11-29T18:03:52.771747+0000","modified":"2022-11-29T18:03:53.874582+0000","last_up_change":"0.000000","last_in_change":"2022-11-29T18:03:53.874582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"228fad84-4a8f-44ce-ba6d-f0433a5a22ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d58467d0-703a-4da6-980d-30bf1051838b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"34a91d7b-b17b-4400-b2f3-0de86be92ac3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c7d3a292-4f7e-456d-ad04-0a67f33eb539","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-29T18:03:54.096 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 10 mgrc shutdown 2022-11-29T18:03:54.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fe6f405d820 msgr2=0x7fe6f40a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fe6f405d820 0x7fe6f40a1470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fe6f40659d0 tx=0x7fe6e4017000).stop 2022-11-29T18:03:54.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 shutdown_connections 2022-11-29T18:03:54.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fe6f4130b70 0x7fe6f40ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6f4093de0 0x7fe6f40987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 --2- 172.21.15.35:0/1247844360 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fe6f405d820 0x7fe6f40a1470 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.099 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 >> 172.21.15.35:0/1247844360 conn(0x7fe6f40624c0 msgr2=0x7fe6f409ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:54.099 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 shutdown_connections 2022-11-29T18:03:54.100 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:03:54.094+0000 7fe6f8851700 1 -- 172.21.15.35:0/1247844360 wait complete. 2022-11-29T18:03:54.109 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-29T18:03:54.109 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-29T18:03:54.379 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:54.373+0000 7fa02c861500 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-29T18:03:54.380 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:54.373+0000 7fca37344500 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-29T18:03:54.438 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:54.438+0000 7f8fe0015500 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-29T18:03:54.513 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-29T18:03:54.522 INFO:tasks.ceph.osd.0.smithi035.stderr:2022-11-29T18:03:54.522+0000 7f80dba7e080 -1 Falling back to public interface 2022-11-29T18:03:54.533 INFO:tasks.ceph.osd.3.smithi038.stderr:2022-11-29T18:03:54.529+0000 7f17344a9080 -1 Falling back to public interface 2022-11-29T18:03:54.533 INFO:tasks.ceph.osd.1.smithi038.stderr:2022-11-29T18:03:54.529+0000 7eff9d859080 -1 Falling back to public interface 2022-11-29T18:03:54.534 INFO:tasks.ceph.osd.2.smithi038.stderr:2022-11-29T18:03:54.529+0000 7fe690b43080 -1 Falling back to public interface 2022-11-29T18:03:54.541 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:54.537+0000 7fca37344500 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-29T18:03:54.542 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:54.537+0000 7fa02c861500 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-29T18:03:54.610 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:54.610+0000 7f8fe0015500 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-29T18:03:54.610 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a8912700 1 Processor -- start 2022-11-29T18:03:54.611 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a8912700 1 -- start start 2022-11-29T18:03:54.613 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a8912700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7ff1a4137d70 0x7ff1a4067a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.613 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a8912700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a4068c00 0x7ff1a4063f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.613 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a8912700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a4137070 con 0x7ff1a409f560 2022-11-29T18:03:54.613 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a8912700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a4138150 con 0x7ff1a4137d70 2022-11-29T18:03:54.613 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a8912700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a409fef0 con 0x7ff1a4068c00 2022-11-29T18:03:54.614 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a259c700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7ff1a4137d70 0x7ff1a4067a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.614 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff19bfff700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a4068c00 0x7ff1a4063f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.614 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a2d9d700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7ff1a409f560 0x7ff1a413a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50756/0 (socket says 172.21.15.35:50756) 2022-11-29T18:03:54.614 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a259c700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7ff1a4137d70 0x7ff1a4067a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:38972/0 (socket says 172.21.15.35:38972) 2022-11-29T18:03:54.615 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a2d9d700 1 -- 172.21.15.35:0/2443113181 learned_addr learned my addr 172.21.15.35:0/2443113181 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:03:54.615 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff19bfff700 1 -- 172.21.15.35:0/2443113181 >> v2:172.21.15.35:3300/0 conn(0x7ff1a4137d70 msgr2=0x7ff1a4067a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:03:54.615 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff19bfff700 1 --2- 172.21.15.35:0/2443113181 >> v2:172.21.15.35:3300/0 conn(0x7ff1a4137d70 0x7ff1a4067a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.615 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff19bfff700 1 -- 172.21.15.35:0/2443113181 >> v1:172.21.15.35:6789/0 conn(0x7ff1a409f560 legacy=0x7ff1a413a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.615 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff19bfff700 1 -- 172.21.15.35:0/2443113181 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a41372c0 con 0x7ff1a4068c00 2022-11-29T18:03:54.616 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff19bfff700 1 --2- 172.21.15.35:0/2443113181 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a4068c00 0x7ff1a4063f70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7ff18c009760 tx=0x7ff18c009060).ready entity=mon.2 client_cookie=563e0d0dae9f4f1c server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:54.616 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a1d9b700 1 -- 172.21.15.35:0/2443113181 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00b590 con 0x7ff1a4068c00 2022-11-29T18:03:54.616 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a1d9b700 1 -- 172.21.15.35:0/2443113181 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c00b6f0 con 0x7ff1a4068c00 2022-11-29T18:03:54.616 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.610+0000 7ff1a1d9b700 1 -- 172.21.15.35:0/2443113181 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00ba70 con 0x7ff1a4068c00 2022-11-29T18:03:54.620 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- 172.21.15.35:0/2443113181 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a4068c00 msgr2=0x7ff1a4063f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.620 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 --2- 172.21.15.35:0/2443113181 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a4068c00 0x7ff1a4063f70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7ff18c009760 tx=0x7ff18c009060).stop 2022-11-29T18:03:54.620 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- 172.21.15.35:0/2443113181 shutdown_connections 2022-11-29T18:03:54.621 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 --2- 172.21.15.35:0/2443113181 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a4068c00 0x7ff1a4063f70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.621 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 --2- 172.21.15.35:0/2443113181 >> v2:172.21.15.35:3300/0 conn(0x7ff1a4137d70 0x7ff1a4067a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.621 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- 172.21.15.35:0/2443113181 >> 172.21.15.35:0/2443113181 conn(0x7ff1a40ad740 msgr2=0x7ff1a40adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:54.621 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- 172.21.15.35:0/2443113181 shutdown_connections 2022-11-29T18:03:54.621 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- 172.21.15.35:0/2443113181 wait complete. 2022-11-29T18:03:54.622 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 Processor -- start 2022-11-29T18:03:54.622 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- start start 2022-11-29T18:03:54.622 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff1a4068c00 0x7ff1a414d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.622 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a409f560 0x7ff1a4091080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.623 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a4137d70 0x7ff1a408f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:03:54.623 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4090c30 con 0x7ff1a4068c00 2022-11-29T18:03:54.623 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40986d0 con 0x7ff1a4137d70 2022-11-29T18:03:54.623 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a8912700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4091570 con 0x7ff1a409f560 2022-11-29T18:03:54.623 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a259c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff1a4068c00 0x7ff1a414d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.624 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a259c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff1a4068c00 0x7ff1a414d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:38980/0 (socket says 172.21.15.35:38980) 2022-11-29T18:03:54.624 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff19bfff700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a409f560 0x7ff1a4091080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.624 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a259c700 1 -- 172.21.15.35:0/46431964 learned_addr learned my addr 172.21.15.35:0/46431964 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:03:54.624 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a2d9d700 1 --2- 172.21.15.35:0/46431964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a4137d70 0x7ff1a408f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:03:54.625 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a259c700 1 -- 172.21.15.35:0/46431964 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a409f560 msgr2=0x7ff1a4091080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.625 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a259c700 1 --2- 172.21.15.35:0/46431964 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a409f560 0x7ff1a4091080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.625 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a259c700 1 -- 172.21.15.35:0/46431964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a4137d70 msgr2=0x7ff1a408f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.625 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a259c700 1 --2- 172.21.15.35:0/46431964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a4137d70 0x7ff1a408f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.625 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.614+0000 7ff1a259c700 1 -- 172.21.15.35:0/46431964 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c0092a0 con 0x7ff1a4068c00 2022-11-29T18:03:54.626 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff1a259c700 1 --2- 172.21.15.35:0/46431964 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff1a4068c00 0x7ff1a414d760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7ff19400e960 tx=0x7ff194009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:03:54.626 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 1 -- 172.21.15.35:0/46431964 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194012600 con 0x7ff1a4068c00 2022-11-29T18:03:54.626 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 1 -- 172.21.15.35:0/46431964 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194012760 con 0x7ff1a4068c00 2022-11-29T18:03:54.626 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a414dd30 con 0x7ff1a4068c00 2022-11-29T18:03:54.626 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 1 -- 172.21.15.35:0/46431964 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194012ae0 con 0x7ff1a4068c00 2022-11-29T18:03:54.627 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a40903a0 con 0x7ff1a4068c00 2022-11-29T18:03:54.627 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 1 -- 172.21.15.35:0/46431964 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 1) v1 ==== 491+0+0 (secure 0 0 0) 0x7ff194022070 con 0x7ff1a4068c00 2022-11-29T18:03:54.627 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2022-11-29T18:03:54.627 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 4 mgrc handle_mgr_map Got map version 1 2022-11-29T18:03:54.628 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 4 mgrc handle_mgr_map Active mgr is now 2022-11-29T18:03:54.628 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 4 mgrc reconnect No active mgr available yet 2022-11-29T18:03:54.628 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff19a7fc700 1 -- 172.21.15.35:0/46431964 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7ff19401a3b0 con 0x7ff1a4068c00 2022-11-29T18:03:54.628 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.618+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a406b650 con 0x7ff1a4068c00 2022-11-29T18:03:54.629 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.622+0000 7ff19a7fc700 1 -- 172.21.15.35:0/46431964 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73156 (secure 0 0 0) 0x7ff19403ade0 con 0x7ff1a4068c00 2022-11-29T18:03:54.688 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.686+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1a4069890 con 0x7ff1a4068c00 2022-11-29T18:03:54.689 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.686+0000 7ff19a7fc700 1 -- 172.21.15.35:0/46431964 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4140 (secure 0 0 0) 0x7ff194029030 con 0x7ff1a4068c00 2022-11-29T18:03:54.689 INFO:teuthology.misc.health.smithi035.stdout: 2022-11-29T18:03:54.689 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":6,"fsid":"338b2b5b-2e59-4e89-9321-f2b909af04fd","created":"2022-11-29T18:03:52.771747+0000","modified":"2022-11-29T18:03:53.874582+0000","last_up_change":"0.000000","last_in_change":"2022-11-29T18:03:53.874582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"228fad84-4a8f-44ce-ba6d-f0433a5a22ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d58467d0-703a-4da6-980d-30bf1051838b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"34a91d7b-b17b-4400-b2f3-0de86be92ac3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c7d3a292-4f7e-456d-ad04-0a67f33eb539","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-29T18:03:54.691 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 10 mgrc shutdown 2022-11-29T18:03:54.691 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff1a4068c00 msgr2=0x7ff1a414d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:03:54.692 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 --2- 172.21.15.35:0/46431964 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff1a4068c00 0x7ff1a414d760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7ff19400e960 tx=0x7ff194009060).stop 2022-11-29T18:03:54.692 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 shutdown_connections 2022-11-29T18:03:54.692 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 --2- 172.21.15.35:0/46431964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1a4137d70 0x7ff1a408f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.692 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 --2- 172.21.15.35:0/46431964 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7ff1a409f560 0x7ff1a4091080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.693 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 --2- 172.21.15.35:0/46431964 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7ff1a4068c00 0x7ff1a414d760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:03:54.693 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 >> 172.21.15.35:0/46431964 conn(0x7ff1a40ad740 msgr2=0x7ff1a40aba00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:03:54.693 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 shutdown_connections 2022-11-29T18:03:54.693 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:03:54.690+0000 7ff1a8912700 1 -- 172.21.15.35:0/46431964 wait complete. 2022-11-29T18:03:54.705 DEBUG:teuthology.misc:0 of 4 OSDs are up 2022-11-29T18:03:54.775 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:54.769+0000 7fca37344500 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-29T18:03:54.776 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:54.769+0000 7fa02c861500 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-29T18:03:54.854 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:54.854+0000 7f8fe0015500 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-29T18:03:54.937 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:54.933+0000 7fca37344500 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-29T18:03:54.939 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:54.933+0000 7fa02c861500 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-29T18:03:54.987 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:54.981+0000 7fa02c861500 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-29T18:03:54.990 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:54.985+0000 7fca37344500 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-29T18:03:55.024 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:55.022+0000 7f8fe0015500 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-29T18:03:55.052 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:55.049+0000 7fa02c861500 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-29T18:03:55.055 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:55.049+0000 7fca37344500 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-29T18:03:55.078 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:55.078+0000 7f8fe0015500 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-29T18:03:55.110 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:55.105+0000 7fa02c861500 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-29T18:03:55.113 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:55.109+0000 7fca37344500 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-29T18:03:55.144 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:55.142+0000 7f8fe0015500 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-29T18:03:55.201 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:55.198+0000 7f8fe0015500 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-29T18:03:55.415 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:55.409+0000 7fa02c861500 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-29T18:03:55.417 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:55.413+0000 7fca37344500 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-29T18:03:55.497 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:55.494+0000 7f8fe0015500 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-29T18:03:55.561 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:55.557+0000 7fa02c861500 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-29T18:03:55.562 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:55.557+0000 7fca37344500 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-29T18:03:55.668 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:55.666+0000 7f8fe0015500 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-29T18:03:55.672 DEBUG:teuthology.orchestra.run.smithi035:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-29T18:03:55.675 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-29T18:03:55.816 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:55.813+0000 7fa02c861500 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-29T18:03:55.817 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:55.813+0000 7fca37344500 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-29T18:03:55.917 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:55.914+0000 7f8fe0015500 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-29T18:03:55.988 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:55.985+0000 7fa02c861500 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-29T18:03:55.989 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:55.985+0000 7fca37344500 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-29T18:03:56.104 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:56.102+0000 7f8fe0015500 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-29T18:03:56.114 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:56.109+0000 7fca37344500 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-29T18:03:56.115 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:56.109+0000 7fa02c861500 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-29T18:03:56.168 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:56.165+0000 7fca37344500 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-29T18:03:56.169 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:56.165+0000 7fa02c861500 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-29T18:03:56.224 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:56.222+0000 7f8fe0015500 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-29T18:03:56.281 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:56.278+0000 7f8fe0015500 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-29T18:03:56.316 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:56.313+0000 7fa02c861500 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-29T18:03:56.368 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:56.365+0000 7fca37344500 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-29T18:03:56.413 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:56.409+0000 7fa02c861500 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-29T18:03:56.430 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:56.430+0000 7f8fe0015500 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-29T18:03:56.465 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:56.461+0000 7fca37344500 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-29T18:03:56.527 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:56.526+0000 7f8fe0015500 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-29T18:03:56.690 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:56.685+0000 7fa02c861500 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-29T18:03:56.707 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:56.701+0000 7fca37344500 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-29T18:03:56.743 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:56.737+0000 7fa02c861500 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-29T18:03:56.759 INFO:tasks.ceph.mgr.y.smithi038.stderr:2022-11-29T18:03:56.753+0000 7fca37344500 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-29T18:03:56.813 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:56.810+0000 7f8fe0015500 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-29T18:03:56.872 INFO:tasks.ceph.mgr.x.smithi035.stderr:2022-11-29T18:03:56.870+0000 7f8fe0015500 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-29T18:03:57.785 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:57.781+0000 7fa01659e700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-11-29T18:03:57.792 INFO:tasks.ceph.mgr.z.smithi038.stderr:2022-11-29T18:03:57.789+0000 7fa01659e700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-11-29T18:03:58.095 INFO:tasks.ceph.osd.3.smithi038.stderr:2022-11-29T18:03:58.089+0000 7f17344a9080 -1 osd.3 0 log_to_monitors {default=true} 2022-11-29T18:03:58.140 INFO:tasks.ceph.osd.1.smithi038.stderr:2022-11-29T18:03:58.137+0000 7eff9d859080 -1 osd.1 0 log_to_monitors {default=true} 2022-11-29T18:03:58.146 INFO:tasks.ceph.osd.0.smithi035.stderr:2022-11-29T18:03:58.146+0000 7f80dba7e080 -1 osd.0 0 log_to_monitors {default=true} 2022-11-29T18:03:58.214 INFO:tasks.ceph.osd.2.smithi038.stderr:2022-11-29T18:03:58.209+0000 7fe690b43080 -1 osd.2 0 log_to_monitors {default=true} 2022-11-29T18:03:59.802 INFO:tasks.ceph.osd.0.smithi035.stderr:2022-11-29T18:03:59.802+0000 7f80d721f700 -1 osd.0 0 waiting for initial osdmap 2022-11-29T18:03:59.809 INFO:tasks.ceph.osd.3.smithi038.stderr:2022-11-29T18:03:59.805+0000 7f172fc4a700 -1 osd.3 0 waiting for initial osdmap 2022-11-29T18:03:59.810 INFO:tasks.ceph.osd.2.smithi038.stderr:2022-11-29T18:03:59.805+0000 7fe68c2e4700 -1 osd.2 0 waiting for initial osdmap 2022-11-29T18:03:59.810 INFO:tasks.ceph.osd.1.smithi038.stderr:2022-11-29T18:03:59.805+0000 7eff98ffa700 -1 osd.1 0 waiting for initial osdmap 2022-11-29T18:03:59.811 INFO:tasks.ceph.osd.0.smithi035.stderr:2022-11-29T18:03:59.810+0000 7f80d3f93700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-29T18:03:59.820 INFO:tasks.ceph.osd.3.smithi038.stderr:2022-11-29T18:03:59.817+0000 7f172c9be700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-29T18:03:59.820 INFO:tasks.ceph.osd.2.smithi038.stderr:2022-11-29T18:03:59.817+0000 7fe689058700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-29T18:03:59.821 INFO:tasks.ceph.osd.1.smithi038.stderr:2022-11-29T18:03:59.817+0000 7eff95d6e700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-29T18:04:01.110 DEBUG:teuthology.orchestra.run.smithi035:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-29T18:04:01.211 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f5321c86700 1 Processor -- start 2022-11-29T18:04:01.211 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f5321c86700 1 -- start start 2022-11-29T18:04:01.215 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f5321c86700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f531c063aa0 0x7f531c097150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.215 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f5321c86700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f531c096a10 0x7f531c098140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.215 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f5321c86700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c05d6e0 0x7f531c13a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.216 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f5321c86700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c097c20 con 0x7f531c063aa0 2022-11-29T18:04:01.216 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f5321c86700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c0979b0 con 0x7f531c096a10 2022-11-29T18:04:01.216 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f5321c86700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c098940 con 0x7f531c05d6e0 2022-11-29T18:04:01.216 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f531c063aa0 0x7f531c097150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.217 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f531c063aa0 0x7f531c097150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39010/0 (socket says 172.21.15.35:39010) 2022-11-29T18:04:01.217 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 -- 172.21.15.35:0/738094207 learned_addr learned my addr 172.21.15.35:0/738094207 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:01.217 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531bfff700 1 --2- 172.21.15.35:0/738094207 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c05d6e0 0x7f531c13a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.217 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531affd700 1 --2- 172.21.15.35:0/738094207 >> v2:172.21.15.38:3300/0 conn(0x7f531c096a10 0x7f531c098140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.217 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 -- 172.21.15.35:0/738094207 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c05d6e0 msgr2=0x7f531c13a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:01.218 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 --2- 172.21.15.35:0/738094207 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c05d6e0 0x7f531c13a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.218 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 -- 172.21.15.35:0/738094207 >> v2:172.21.15.38:3300/0 conn(0x7f531c096a10 msgr2=0x7f531c098140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:01.218 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 --2- 172.21.15.35:0/738094207 >> v2:172.21.15.38:3300/0 conn(0x7f531c096a10 0x7f531c098140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.218 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 -- 172.21.15.35:0/738094207 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c063e80 con 0x7f531c063aa0 2022-11-29T18:04:01.219 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531b7fe700 1 --2- 172.21.15.35:0/738094207 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c063aa0 0x7f531c097150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f531c09f8d0 tx=0x7f530c0090f0).ready entity=mon.0 client_cookie=c850cc29bfd79e75 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.219 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531a7fc700 1 -- 172.21.15.35:0/738094207 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00f510 con 0x7f531c063aa0 2022-11-29T18:04:01.219 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531a7fc700 1 -- 172.21.15.35:0/738094207 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c00f670 con 0x7f531c063aa0 2022-11-29T18:04:01.219 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.210+0000 7f531a7fc700 1 -- 172.21.15.35:0/738094207 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00fa10 con 0x7f531c063aa0 2022-11-29T18:04:01.220 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 -- 172.21.15.35:0/738094207 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c063aa0 msgr2=0x7f531c097150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.220 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 --2- 172.21.15.35:0/738094207 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c063aa0 0x7f531c097150 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f531c09f8d0 tx=0x7f530c0090f0).stop 2022-11-29T18:04:01.220 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 -- 172.21.15.35:0/738094207 shutdown_connections 2022-11-29T18:04:01.221 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 --2- 172.21.15.35:0/738094207 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c05d6e0 0x7f531c13a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.221 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 --2- 172.21.15.35:0/738094207 >> v2:172.21.15.38:3300/0 conn(0x7f531c096a10 0x7f531c098140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.221 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 --2- 172.21.15.35:0/738094207 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c063aa0 0x7f531c097150 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.222 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 -- 172.21.15.35:0/738094207 >> 172.21.15.35:0/738094207 conn(0x7f531c0619f0 msgr2=0x7f531c061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:01.222 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 -- 172.21.15.35:0/738094207 shutdown_connections 2022-11-29T18:04:01.222 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 -- 172.21.15.35:0/738094207 wait complete. 2022-11-29T18:04:01.226 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 Processor -- start 2022-11-29T18:04:01.226 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.214+0000 7f5321c86700 1 -- start start 2022-11-29T18:04:01.226 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f531c05d6e0 0x7f531c0ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.227 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c063aa0 0x7f531c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.227 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c096a10 0x7f531c09ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.227 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c14a410 con 0x7f531c096a10 2022-11-29T18:04:01.227 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c14a6d0 con 0x7f531c05d6e0 2022-11-29T18:04:01.228 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c09c180 con 0x7f531c063aa0 2022-11-29T18:04:01.228 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c063aa0 0x7f531c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.228 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531bfff700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c096a10 0x7f531c09ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.228 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c063aa0 0x7f531c149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:36920/0 (socket says 172.21.15.35:36920) 2022-11-29T18:04:01.229 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531b7fe700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f531c05d6e0 0x7f531c0ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.229 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531bfff700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c096a10 0x7f531c09ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:39020/0 (socket says 172.21.15.35:39020) 2022-11-29T18:04:01.229 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 -- 172.21.15.35:0/2866712925 learned_addr learned my addr 172.21.15.35:0/2866712925 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:01.229 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 -- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f531c05d6e0 msgr2=0x7f531c0ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.230 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f531c05d6e0 0x7f531c0ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.230 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 -- 172.21.15.35:0/2866712925 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c096a10 msgr2=0x7f531c09ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.230 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c096a10 0x7f531c09ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.231 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 -- 172.21.15.35:0/2866712925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c009370 con 0x7f531c063aa0 2022-11-29T18:04:01.231 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f531affd700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c063aa0 0x7f531c149bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f530400e900 tx=0x7f5304009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.231 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f53137fe700 1 -- 172.21.15.35:0/2866712925 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304012600 con 0x7f531c063aa0 2022-11-29T18:04:01.232 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f53137fe700 1 -- 172.21.15.35:0/2866712925 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5304012760 con 0x7f531c063aa0 2022-11-29T18:04:01.232 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c14a0b0 con 0x7f531c063aa0 2022-11-29T18:04:01.232 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f53137fe700 1 -- 172.21.15.35:0/2866712925 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304012ae0 con 0x7f531c063aa0 2022-11-29T18:04:01.232 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c0ad8d0 con 0x7f531c063aa0 2022-11-29T18:04:01.232 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.218+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c06b650 con 0x7f531c063aa0 2022-11-29T18:04:01.233 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f53137fe700 1 -- 172.21.15.35:0/2866712925 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f5304022070 con 0x7f531c063aa0 2022-11-29T18:04:01.233 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f53137fe700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:01.233 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f53137fe700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:01.233 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f53137fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:01.234 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f53137fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:01.234 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f53137fe700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5300084de0 0x7f53000872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.234 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f53137fe700 1 -- 172.21.15.35:0/2866712925 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2772+0+0 (secure 0 0 0) 0x7f53040a72a0 con 0x7f531c063aa0 2022-11-29T18:04:01.235 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f531b7fe700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5300084de0 0x7f53000872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.235 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.222+0000 7f531b7fe700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5300084de0 0x7f53000872a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f531c09f8d0 tx=0x7f530c017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.236 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.230+0000 7f53137fe700 1 -- 172.21.15.35:0/2866712925 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f530407ba50 con 0x7f531c063aa0 2022-11-29T18:04:01.385 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.382+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f531c098630 con 0x7f531c063aa0 2022-11-29T18:04:01.386 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.386+0000 7f53137fe700 1 -- 172.21.15.35:0/2866712925 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) v1 ==== 74+0+5815 (secure 0 0 0) 0x7f530407f070 con 0x7f531c063aa0 2022-11-29T18:04:01.386 INFO:teuthology.misc.health.smithi035.stdout: 2022-11-29T18:04:01.387 INFO:teuthology.misc.health.smithi035.stdout:{"epoch":9,"fsid":"338b2b5b-2e59-4e89-9321-f2b909af04fd","created":"2022-11-29T18:03:52.771747+0000","modified":"2022-11-29T18:04:00.802110+0000","last_up_change":"2022-11-29T18:04:00.802110+0000","last_in_change":"2022-11-29T18:03:53.874582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"228fad84-4a8f-44ce-ba6d-f0433a5a22ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":22147},{"type":"v1","addr":"172.21.15.35:6801","nonce":22147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":22147},{"type":"v1","addr":"172.21.15.35:6803","nonce":22147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":22147},{"type":"v1","addr":"172.21.15.35:6807","nonce":22147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":22147},{"type":"v1","addr":"172.21.15.35:6805","nonce":22147}]},"public_addr":"172.21.15.35:6801/22147","cluster_addr":"172.21.15.35:6803/22147","heartbeat_back_addr":"172.21.15.35:6807/22147","heartbeat_front_addr":"172.21.15.35:6805/22147","state":["exists","up"]},{"osd":1,"uuid":"d58467d0-703a-4da6-980d-30bf1051838b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24521},{"type":"v1","addr":"172.21.15.38:6803","nonce":24521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6805","nonce":24521},{"type":"v1","addr":"172.21.15.38:6806","nonce":24521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6811","nonce":24521},{"type":"v1","addr":"172.21.15.38:6815","nonce":24521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24521},{"type":"v1","addr":"172.21.15.38:6809","nonce":24521}]},"public_addr":"172.21.15.38:6803/24521","cluster_addr":"172.21.15.38:6806/24521","heartbeat_back_addr":"172.21.15.38:6815/24521","heartbeat_front_addr":"172.21.15.38:6809/24521","state":["exists","up"]},{"osd":2,"uuid":"34a91d7b-b17b-4400-b2f3-0de86be92ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6813","nonce":24523},{"type":"v1","addr":"172.21.15.38:6816","nonce":24523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24523},{"type":"v1","addr":"172.21.15.38:6819","nonce":24523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24523},{"type":"v1","addr":"172.21.15.38:6823","nonce":24523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24523},{"type":"v1","addr":"172.21.15.38:6821","nonce":24523}]},"public_addr":"172.21.15.38:6816/24523","cluster_addr":"172.21.15.38:6819/24523","heartbeat_back_addr":"172.21.15.38:6823/24523","heartbeat_front_addr":"172.21.15.38:6821/24523","state":["exists","up"]},{"osd":3,"uuid":"c7d3a292-4f7e-456d-ad04-0a67f33eb539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24522},{"type":"v1","addr":"172.21.15.38:6801","nonce":24522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24522},{"type":"v1","addr":"172.21.15.38:6807","nonce":24522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24522},{"type":"v1","addr":"172.21.15.38:6817","nonce":24522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24522},{"type":"v1","addr":"172.21.15.38:6812","nonce":24522}]},"public_addr":"172.21.15.38:6801/24522","cluster_addr":"172.21.15.38:6807/24522","heartbeat_back_addr":"172.21.15.38:6817/24522","heartbeat_front_addr":"172.21.15.38:6812/24522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-29T18:04:01.390 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 10 mgrc shutdown 2022-11-29T18:04:01.391 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5300084de0 msgr2=0x7f53000872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.391 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5300084de0 0x7f53000872a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f531c09f8d0 tx=0x7f530c017000).stop 2022-11-29T18:04:01.391 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c063aa0 msgr2=0x7f531c149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.392 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c063aa0 0x7f531c149bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f530400e900 tx=0x7f5304009060).stop 2022-11-29T18:04:01.392 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 shutdown_connections 2022-11-29T18:04:01.392 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5300084de0 0x7f53000872a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.392 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f531c096a10 0x7f531c09ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.393 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f531c063aa0 0x7f531c149bc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.393 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 --2- 172.21.15.35:0/2866712925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f531c05d6e0 0x7f531c0ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.393 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 >> 172.21.15.35:0/2866712925 conn(0x7f531c0619f0 msgr2=0x7f531c06abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:01.394 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 shutdown_connections 2022-11-29T18:04:01.394 INFO:teuthology.misc.health.smithi035.stderr:2022-11-29T18:04:01.390+0000 7f5321c86700 1 -- 172.21.15.35:0/2866712925 wait complete. 2022-11-29T18:04:01.404 DEBUG:teuthology.misc:4 of 4 OSDs are up 2022-11-29T18:04:01.404 INFO:tasks.ceph:Creating RBD pool 2022-11-29T18:04:01.405 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-29T18:04:01.510 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb058a41700 1 Processor -- start 2022-11-29T18:04:01.510 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb058a41700 1 -- start start 2022-11-29T18:04:01.511 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb058a41700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb05409eb10 0x7fb05413e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.511 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb058a41700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05409e6e0 con 0x7fb05413e500 2022-11-29T18:04:01.511 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb058a41700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05409a3b0 con 0x7fb054063880 2022-11-29T18:04:01.512 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb058a41700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05409a070 con 0x7fb05409eb10 2022-11-29T18:04:01.512 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb052ffd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb05409eb10 0x7fb05413e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.512 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb053fff700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fb05413e500 0x7fb054144f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50772/0 (socket says 172.21.15.35:50772) 2022-11-29T18:04:01.512 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.506+0000 7fb053fff700 1 -- 172.21.15.35:0/1766537756 learned_addr learned my addr 172.21.15.35:0/1766537756 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:01.513 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb052ffd700 1 -- 172.21.15.35:0/1766537756 >> v1:172.21.15.38:6789/0 conn(0x7fb054063880 legacy=0x7fb05409ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.513 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb052ffd700 1 -- 172.21.15.35:0/1766537756 >> v1:172.21.15.35:6789/0 conn(0x7fb05413e500 legacy=0x7fb054144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.513 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb052ffd700 1 -- 172.21.15.35:0/1766537756 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05409b5e0 con 0x7fb05409eb10 2022-11-29T18:04:01.513 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb052ffd700 1 --2- 172.21.15.35:0/1766537756 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb05409eb10 0x7fb05413e010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fb044009e90 tx=0x7fb044009780).ready entity=mon.2 client_cookie=ab11910c70500fc7 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.514 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb0527fc700 1 -- 172.21.15.35:0/1766537756 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400e070 con 0x7fb05409eb10 2022-11-29T18:04:01.514 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb0527fc700 1 -- 172.21.15.35:0/1766537756 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04400bc30 con 0x7fb05409eb10 2022-11-29T18:04:01.514 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb0527fc700 1 -- 172.21.15.35:0/1766537756 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044012410 con 0x7fb05409eb10 2022-11-29T18:04:01.520 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 -- 172.21.15.35:0/1766537756 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb05409eb10 msgr2=0x7fb05413e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.520 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 --2- 172.21.15.35:0/1766537756 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb05409eb10 0x7fb05413e010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fb044009e90 tx=0x7fb044009780).stop 2022-11-29T18:04:01.521 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 -- 172.21.15.35:0/1766537756 shutdown_connections 2022-11-29T18:04:01.521 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 --2- 172.21.15.35:0/1766537756 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb05409eb10 0x7fb05413e010 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.522 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 -- 172.21.15.35:0/1766537756 >> 172.21.15.35:0/1766537756 conn(0x7fb0540acf70 msgr2=0x7fb054062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:01.522 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 -- 172.21.15.35:0/1766537756 shutdown_connections 2022-11-29T18:04:01.522 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 -- 172.21.15.35:0/1766537756 wait complete. 2022-11-29T18:04:01.523 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 Processor -- start 2022-11-29T18:04:01.523 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.510+0000 7fb058a41700 1 -- start start 2022-11-29T18:04:01.523 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb054063880 0x7fb0540a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.524 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb05409eb10 0x7fb0540a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.524 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb05413e500 0x7fb0540a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.524 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540a1ef0 con 0x7fb05409eb10 2022-11-29T18:04:01.524 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb054097a70 con 0x7fb05413e500 2022-11-29T18:04:01.525 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05414adc0 con 0x7fb054063880 2022-11-29T18:04:01.525 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb054063880 0x7fb0540a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.525 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb054063880 0x7fb0540a26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:36934/0 (socket says 172.21.15.35:36934) 2022-11-29T18:04:01.525 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 -- 172.21.15.35:0/6085403 learned_addr learned my addr 172.21.15.35:0/6085403 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:01.525 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb052ffd700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb05409eb10 0x7fb0540a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.526 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb053fff700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb05413e500 0x7fb0540a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.526 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 -- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb05413e500 msgr2=0x7fb0540a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.527 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb05413e500 0x7fb0540a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.527 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 -- 172.21.15.35:0/6085403 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb05409eb10 msgr2=0x7fb0540a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.527 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb05409eb10 0x7fb0540a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.527 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 -- 172.21.15.35:0/6085403 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044009a20 con 0x7fb054063880 2022-11-29T18:04:01.528 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb053fff700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb05413e500 0x7fb0540a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2022-11-29T18:04:01.528 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb0537fe700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb054063880 0x7fb0540a26d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fb04c00c4c0 tx=0x7fb04c007e00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.529 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb03b7fe700 1 -- 172.21.15.35:0/6085403 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c007020 con 0x7fb054063880 2022-11-29T18:04:01.529 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb03b7fe700 1 -- 172.21.15.35:0/6085403 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c001e90 con 0x7fb054063880 2022-11-29T18:04:01.530 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05414b080 con 0x7fb054063880 2022-11-29T18:04:01.530 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb03b7fe700 1 -- 172.21.15.35:0/6085403 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0073b0 con 0x7fb054063880 2022-11-29T18:04:01.530 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0540a21d0 con 0x7fb054063880 2022-11-29T18:04:01.531 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.514+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb054046830 con 0x7fb054063880 2022-11-29T18:04:01.531 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb03b7fe700 1 -- 172.21.15.35:0/6085403 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7fb04c010070 con 0x7fb054063880 2022-11-29T18:04:01.532 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb03b7fe700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:01.532 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb03b7fe700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:01.533 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb03b7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:01.533 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb03b7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:01.534 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb03b7fe700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb034084de0 0x7fb0340872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.534 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb03b7fe700 1 -- 172.21.15.35:0/6085403 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2772+0+0 (secure 0 0 0) 0x7fb04c0a6950 con 0x7fb054063880 2022-11-29T18:04:01.534 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb052ffd700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb034084de0 0x7fb0340872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.535 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.518+0000 7fb052ffd700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb034084de0 0x7fb0340872a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fb0440097c0 tx=0x7fb05409b770).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.536 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.522+0000 7fb03b7fe700 1 -- 172.21.15.35:0/6085403 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7fb04c07b100 con 0x7fb054063880 2022-11-29T18:04:01.685 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.682+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fb05409df60 con 0x7fb054063880 2022-11-29T18:04:01.835 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.834+0000 7fb03b7fe700 1 -- 172.21.15.35:0/6085403 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v10) v1 ==== 109+0+0 (secure 0 0 0) 0x7fb04c07e720 con 0x7fb054063880 2022-11-29T18:04:01.840 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 10 mgrc shutdown 2022-11-29T18:04:01.840 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb034084de0 msgr2=0x7fb0340872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.840 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb034084de0 0x7fb0340872a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fb0440097c0 tx=0x7fb05409b770).stop 2022-11-29T18:04:01.841 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb054063880 msgr2=0x7fb0540a26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.841 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb054063880 0x7fb0540a26d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fb04c00c4c0 tx=0x7fb04c007e00).stop 2022-11-29T18:04:01.841 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 shutdown_connections 2022-11-29T18:04:01.841 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb034084de0 0x7fb0340872a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.841 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb05413e500 0x7fb0540a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.842 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb05409eb10 0x7fb0540a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.842 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 --2- 172.21.15.35:0/6085403 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb054063880 0x7fb0540a26d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.842 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 >> 172.21.15.35:0/6085403 conn(0x7fb0540acf70 msgr2=0x7fb05413dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:01.842 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 shutdown_connections 2022-11-29T18:04:01.843 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.838+0000 7fb058a41700 1 -- 172.21.15.35:0/6085403 wait complete. 2022-11-29T18:04:01.843 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'rbd' created 2022-11-29T18:04:01.855 DEBUG:teuthology.orchestra.run.smithi035:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-29T18:04:01.960 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.958+0000 7f899ac04700 1 Processor -- start 2022-11-29T18:04:01.961 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.958+0000 7f899ac04700 1 -- start start 2022-11-29T18:04:01.967 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940aea10 0x7f89940abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.968 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f899409f1e0 0x7f8994063e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.969 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899413df70 con 0x7f89940ae630 2022-11-29T18:04:01.970 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899409f690 con 0x7f899409f1e0 2022-11-29T18:04:01.970 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899413e1f0 con 0x7f89940aea10 2022-11-29T18:04:01.971 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8999401700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940aea10 0x7f89940abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.971 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8999401700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940aea10 0x7f89940abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:36950/0 (socket says 172.21.15.35:36950) 2022-11-29T18:04:01.971 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8999401700 1 -- 172.21.15.35:0/346101787 learned_addr learned my addr 172.21.15.35:0/346101787 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:01.972 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899a403700 1 --2- 172.21.15.35:0/346101787 >> v2:172.21.15.38:3300/0 conn(0x7f899409f1e0 0x7f8994063e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.972 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8998c00700 1 -- 172.21.15.35:0/346101787 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662366599 0 0) 0x7f899413df70 con 0x7f89940ae630 2022-11-29T18:04:01.973 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8998c00700 1 -- 172.21.15.35:0/346101787 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f897c0033f0 con 0x7f89940ae630 2022-11-29T18:04:01.973 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8999401700 1 -- 172.21.15.35:0/346101787 >> v2:172.21.15.38:3300/0 conn(0x7f899409f1e0 msgr2=0x7f8994063e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:01.973 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8999401700 1 --2- 172.21.15.35:0/346101787 >> v2:172.21.15.38:3300/0 conn(0x7f899409f1e0 0x7f8994063e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.974 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8999401700 1 -- 172.21.15.35:0/346101787 >> v1:172.21.15.38:6789/0 conn(0x7f89940ae630 legacy=0x7f899413d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.974 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8999401700 1 -- 172.21.15.35:0/346101787 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899413e4b0 con 0x7f89940aea10 2022-11-29T18:04:01.975 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899a403700 1 --2- 172.21.15.35:0/346101787 >> v2:172.21.15.38:3300/0 conn(0x7f899409f1e0 0x7f8994063e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-29T18:04:01.975 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8999401700 1 --2- 172.21.15.35:0/346101787 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940aea10 0x7f89940abe60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f8990006fd0 tx=0x7f89900090f0).ready entity=mon.2 client_cookie=f8ffa24749f350b7 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.975 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8998c00700 1 -- 172.21.15.35:0/346101787 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000f510 con 0x7f89940aea10 2022-11-29T18:04:01.976 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8998c00700 1 -- 172.21.15.35:0/346101787 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000f670 con 0x7f89940aea10 2022-11-29T18:04:01.976 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f8998c00700 1 -- 172.21.15.35:0/346101787 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000f9f0 con 0x7f89940aea10 2022-11-29T18:04:01.977 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 -- 172.21.15.35:0/346101787 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940aea10 msgr2=0x7f89940abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.977 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 --2- 172.21.15.35:0/346101787 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940aea10 0x7f89940abe60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f8990006fd0 tx=0x7f89900090f0).stop 2022-11-29T18:04:01.978 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 -- 172.21.15.35:0/346101787 shutdown_connections 2022-11-29T18:04:01.978 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 --2- 172.21.15.35:0/346101787 >> v2:172.21.15.38:3300/0 conn(0x7f899409f1e0 0x7f8994063e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.979 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 --2- 172.21.15.35:0/346101787 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940aea10 0x7f89940abe60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.979 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 -- 172.21.15.35:0/346101787 >> 172.21.15.35:0/346101787 conn(0x7f89940619f0 msgr2=0x7f8994061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:01.980 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 -- 172.21.15.35:0/346101787 shutdown_connections 2022-11-29T18:04:01.980 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.962+0000 7f899ac04700 1 -- 172.21.15.35:0/346101787 wait complete. 2022-11-29T18:04:01.980 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899ac04700 1 Processor -- start 2022-11-29T18:04:01.981 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899ac04700 1 -- start start 2022-11-29T18:04:01.986 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899ac04700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f899409f1e0 0x7f8994138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.986 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899ac04700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940ae630 0x7f89940aa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.986 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899ac04700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8994140620 0x7f89940a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.987 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899ac04700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89941385f0 con 0x7f8994140620 2022-11-29T18:04:01.987 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899ac04700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899409d260 con 0x7f899409f1e0 2022-11-29T18:04:01.987 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899ac04700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8994138f10 con 0x7f89940ae630 2022-11-29T18:04:01.987 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940ae630 0x7f89940aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.988 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940ae630 0x7f89940aa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:36952/0 (socket says 172.21.15.35:36952) 2022-11-29T18:04:01.988 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 -- 172.21.15.35:0/978270237 learned_addr learned my addr 172.21.15.35:0/978270237 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:01.988 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f899a403700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8994140620 0x7f89940a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.988 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999c02700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f899409f1e0 0x7f8994138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.988 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 -- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f899409f1e0 msgr2=0x7f8994138a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.989 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f899409f1e0 0x7f8994138a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.989 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 -- 172.21.15.35:0/978270237 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8994140620 msgr2=0x7f89940a86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:01.989 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8994140620 0x7f89940a86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:01.989 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 -- 172.21.15.35:0/978270237 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c009a20 con 0x7f89940ae630 2022-11-29T18:04:01.990 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8999401700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940ae630 0x7f89940aa190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f899413e120 tx=0x7f8990017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.990 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8985ffb700 1 -- 172.21.15.35:0/978270237 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990006a40 con 0x7f89940ae630 2022-11-29T18:04:01.990 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.966+0000 7f8985ffb700 1 -- 172.21.15.35:0/978270237 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000f7e0 con 0x7f89940ae630 2022-11-29T18:04:01.990 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8990009370 con 0x7f89940ae630 2022-11-29T18:04:01.991 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f8985ffb700 1 -- 172.21.15.35:0/978270237 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899001c670 con 0x7f89940ae630 2022-11-29T18:04:01.991 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899409cd70 con 0x7f89940ae630 2022-11-29T18:04:01.991 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f8985ffb700 1 -- 172.21.15.35:0/978270237 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f899002c070 con 0x7f89940ae630 2022-11-29T18:04:01.991 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f8985ffb700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:01.992 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8994138740 con 0x7f89940ae630 2022-11-29T18:04:01.992 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f8985ffb700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:01.992 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f8985ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:01.992 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f8985ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:01.992 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.970+0000 7f8985ffb700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8974084e10 0x7f89740872d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:01.993 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.974+0000 7f8985ffb700 1 -- 172.21.15.35:0/978270237 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 3114+0+0 (secure 0 0 0) 0x7f89900a8ea0 con 0x7f89940ae630 2022-11-29T18:04:01.993 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.974+0000 7f8999c02700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8974084e10 0x7f89740872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:01.993 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.974+0000 7f8999c02700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8974084e10 0x7f89740872d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8988003930 tx=0x7f8988007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:01.993 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:01.982+0000 7f8985ffb700 1 -- 172.21.15.35:0/978270237 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f8994138740 con 0x7f89940ae630 2022-11-29T18:04:02.137 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.134+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8994138740 con 0x7f89940ae630 2022-11-29T18:04:02.844 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.842+0000 7f8985ffb700 1 -- 172.21.15.35:0/978270237 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v11) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8994138740 con 0x7f89940ae630 2022-11-29T18:04:02.848 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.846+0000 7f899ac04700 10 mgrc shutdown 2022-11-29T18:04:02.851 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8974084e10 msgr2=0x7f89740872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:02.851 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8974084e10 0x7f89740872d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8988003930 tx=0x7f8988007bd0).stop 2022-11-29T18:04:02.852 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940ae630 msgr2=0x7f89940aa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:02.852 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940ae630 0x7f89940aa190 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f899413e120 tx=0x7f8990017000).stop 2022-11-29T18:04:02.852 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 shutdown_connections 2022-11-29T18:04:02.852 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8974084e10 0x7f89740872d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:02.853 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8994140620 0x7f89940a86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:02.853 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f89940ae630 0x7f89940aa190 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:02.853 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 --2- 172.21.15.35:0/978270237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f899409f1e0 0x7f8994138a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:02.853 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 >> 172.21.15.35:0/978270237 conn(0x7f89940619f0 msgr2=0x7f8994061dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:02.853 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 shutdown_connections 2022-11-29T18:04:02.854 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.850+0000 7f899ac04700 1 -- 172.21.15.35:0/978270237 wait complete. 2022-11-29T18:04:02.854 INFO:teuthology.orchestra.run.smithi035.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-29T18:04:02.864 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-29T18:04:02.865 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-29T18:04:02.865 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-29T18:04:02.874 INFO:tasks.ceph.mds.a:Started 2022-11-29T18:04:02.875 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-29T18:04:02.875 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-29T18:04:02.878 INFO:tasks.ceph.mds.c:Started 2022-11-29T18:04:02.879 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-29T18:04:02.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-29T18:04:02.883 INFO:tasks.ceph.mds.b:Started 2022-11-29T18:04:02.884 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-29T18:04:02.884 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-29T18:04:02.947 INFO:tasks.ceph.mds.a.smithi035.stdout:starting mds.a at 2022-11-29T18:04:02.948 INFO:tasks.ceph.mds.c.smithi035.stdout:starting mds.c at 2022-11-29T18:04:02.961 INFO:tasks.ceph.mds.b.smithi038.stdout:starting mds.b at 2022-11-29T18:04:03.001 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.998+0000 7f6baddec700 1 Processor -- start 2022-11-29T18:04:03.001 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:02.998+0000 7f6baddec700 1 -- start start 2022-11-29T18:04:03.008 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f6ba80a2920 0x7f6ba8145c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.009 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba809c680 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.010 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba813b320 con 0x7f6ba80b1390 2022-11-29T18:04:03.010 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba80a2d90 con 0x7f6ba80a2920 2022-11-29T18:04:03.010 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6bad5eb700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f6ba80a2920 0x7f6ba8145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.010 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6bacdea700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f6ba80b0fb0 0x7f6ba813ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50292/0 (socket says 172.21.15.35:50292) 2022-11-29T18:04:03.011 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6bad5eb700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f6ba80a2920 0x7f6ba8145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43752/0 (socket says 172.21.15.35:43752) 2022-11-29T18:04:03.011 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6bacdea700 1 -- 172.21.15.35:0/2751384172 learned_addr learned my addr 172.21.15.35:0/2751384172 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:03.011 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6bad5eb700 1 -- 172.21.15.35:0/2751384172 >> v1:172.21.15.38:6789/0 conn(0x7f6ba80b1390 legacy=0x7f6ba809bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.011 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6bad5eb700 1 -- 172.21.15.35:0/2751384172 >> v1:172.21.15.35:6789/0 conn(0x7f6ba80b0fb0 legacy=0x7f6ba813ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.011 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6ba77fe700 1 -- 172.21.15.35:0/2751384172 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894781594 0 0) 0x7f6ba809c680 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.012 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6bad5eb700 1 -- 172.21.15.35:0/2751384172 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba809ae60 con 0x7f6ba80a2920 2022-11-29T18:04:03.012 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6bad5eb700 1 --2- 172.21.15.35:0/2751384172 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba80a2920 0x7f6ba8145c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f6b98009760 tx=0x7f6b98009060).ready entity=mon.0 client_cookie=a5340d9163ffd58f server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.012 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6ba77fe700 1 -- 172.21.15.35:0/2751384172 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98009120 con 0x7f6ba80a2920 2022-11-29T18:04:03.012 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6ba77fe700 1 -- 172.21.15.35:0/2751384172 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9800b580 con 0x7f6ba80a2920 2022-11-29T18:04:03.013 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6ba77fe700 1 -- 172.21.15.35:0/2751384172 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800b900 con 0x7f6ba80a2920 2022-11-29T18:04:03.013 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 -- 172.21.15.35:0/2751384172 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba80a2920 msgr2=0x7f6ba8145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.013 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 --2- 172.21.15.35:0/2751384172 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba80a2920 0x7f6ba8145c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f6b98009760 tx=0x7f6b98009060).stop 2022-11-29T18:04:03.013 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 -- 172.21.15.35:0/2751384172 shutdown_connections 2022-11-29T18:04:03.014 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 --2- 172.21.15.35:0/2751384172 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba80a2920 0x7f6ba8145c20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.014 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 -- 172.21.15.35:0/2751384172 >> 172.21.15.35:0/2751384172 conn(0x7f6ba8069bf0 msgr2=0x7f6ba80b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:03.014 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 -- 172.21.15.35:0/2751384172 shutdown_connections 2022-11-29T18:04:03.014 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.002+0000 7f6baddec700 1 -- 172.21.15.35:0/2751384172 wait complete. 2022-11-29T18:04:03.015 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 Processor -- start 2022-11-29T18:04:03.015 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 -- start start 2022-11-29T18:04:03.015 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ba80a2920 0x7f6ba8067e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.015 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6ba80b0fb0 0x7f6ba80acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.016 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba8144680 0x7f6ba80acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.016 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81396e0 con 0x7f6ba8144680 2022-11-29T18:04:03.016 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80ad4a0 con 0x7f6ba80a2920 2022-11-29T18:04:03.016 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80ad750 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.016 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6ba80b0fb0 0x7f6ba80acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.017 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6bad5eb700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba8144680 0x7f6ba80acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.017 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6ba80b0fb0 0x7f6ba80acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57724/0 (socket says 172.21.15.35:57724) 2022-11-29T18:04:03.017 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 -- 172.21.15.35:0/1024644914 learned_addr learned my addr 172.21.15.35:0/1024644914 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:03.017 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6bacdea700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ba80a2920 0x7f6ba8067e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.017 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 -- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ba80a2920 msgr2=0x7f6ba8067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.018 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ba80a2920 0x7f6ba8067e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.018 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 -- 172.21.15.35:0/1024644914 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba8144680 msgr2=0x7f6ba80acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.018 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba8144680 0x7f6ba80acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.018 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 -- 172.21.15.35:0/1024644914 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b980092a0 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.019 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6bad5eb700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba8144680 0x7f6ba80acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-29T18:04:03.019 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba7fff700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6ba80b0fb0 0x7f6ba80acac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f6b9c00c4f0 tx=0x7f6b9c007e00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.019 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba4ff9700 1 -- 172.21.15.35:0/1024644914 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00ce60 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.019 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba4ff9700 1 -- 172.21.15.35:0/1024644914 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c001e90 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.020 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba80ada10 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.020 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6ba4ff9700 1 -- 172.21.15.35:0/1024644914 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c007250 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.020 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.006+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba8068370 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.021 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.010+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba80687a0 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.010+0000 7f6ba4ff9700 1 -- 172.21.15.35:0/1024644914 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f6b9c010070 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.010+0000 7f6ba4ff9700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:03.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.014+0000 7f6ba4ff9700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:03.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.014+0000 7f6ba4ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:03.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.014+0000 7f6ba4ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:03.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.014+0000 7f6ba4ff9700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6b88084de0 0x7f6b880872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.014+0000 7f6ba4ff9700 1 -- 172.21.15.35:0/1024644914 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 3517+0+0 (secure 0 0 0) 0x7f6b9c0a6af0 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.014+0000 7f6bacdea700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6b88084de0 0x7f6b880872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.014+0000 7f6bacdea700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6b88084de0 0x7f6b880872a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f6ba0001560 tx=0x7f6ba0007f10).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.022+0000 7f6ba4ff9700 1 -- 172.21.15.35:0/1024644914 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f6b9c07afc0 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.173 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.170+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ba809c8d0 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.174 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.174+0000 7f6ba4ff9700 1 -- 172.21.15.35:0/1024644914 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+9009 (secure 0 0 0) 0x7f6b9c07e5e0 con 0x7f6ba80b0fb0 2022-11-29T18:04:03.175 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:04:03.176 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":11,"fsid":"338b2b5b-2e59-4e89-9321-f2b909af04fd","created":"2022-11-29T18:03:52.771747+0000","modified":"2022-11-29T18:04:02.832051+0000","last_up_change":"2022-11-29T18:04:00.802110+0000","last_in_change":"2022-11-29T18:03:53.874582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"rbd","create_time":"2022-11-29T18:04:01.750436+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"device_health_metrics","create_time":"2022-11-29T18:04:01.898453+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"228fad84-4a8f-44ce-ba6d-f0433a5a22ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":22147},{"type":"v1","addr":"172.21.15.35:6801","nonce":22147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":22147},{"type":"v1","addr":"172.21.15.35:6803","nonce":22147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":22147},{"type":"v1","addr":"172.21.15.35:6807","nonce":22147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":22147},{"type":"v1","addr":"172.21.15.35:6805","nonce":22147}]},"public_addr":"172.21.15.35:6801/22147","cluster_addr":"172.21.15.35:6803/22147","heartbeat_back_addr":"172.21.15.35:6807/22147","heartbeat_front_addr":"172.21.15.35:6805/22147","state":["exists","up"]},{"osd":1,"uuid":"d58467d0-703a-4da6-980d-30bf1051838b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24521},{"type":"v1","addr":"172.21.15.38:6803","nonce":24521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6805","nonce":24521},{"type":"v1","addr":"172.21.15.38:6806","nonce":24521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6811","nonce":24521},{"type":"v1","addr":"172.21.15.38:6815","nonce":24521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24521},{"type":"v1","addr":"172.21.15.38:6809","nonce":24521}]},"public_addr":"172.21.15.38:6803/24521","cluster_addr":"172.21.15.38:6806/24521","heartbeat_back_addr":"172.21.15.38:6815/24521","heartbeat_front_addr":"172.21.15.38:6809/24521","state":["exists","up"]},{"osd":2,"uuid":"34a91d7b-b17b-4400-b2f3-0de86be92ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6813","nonce":24523},{"type":"v1","addr":"172.21.15.38:6816","nonce":24523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24523},{"type":"v1","addr":"172.21.15.38:6819","nonce":24523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24523},{"type":"v1","addr":"172.21.15.38:6823","nonce":24523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24523},{"type":"v1","addr":"172.21.15.38:6821","nonce":24523}]},"public_addr":"172.21.15.38:6816/24523","cluster_addr":"172.21.15.38:6819/24523","heartbeat_back_addr":"172.21.15.38:6823/24523","heartbeat_front_addr":"172.21.15.38:6821/24523","state":["exists","up"]},{"osd":3,"uuid":"c7d3a292-4f7e-456d-ad04-0a67f33eb539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24522},{"type":"v1","addr":"172.21.15.38:6801","nonce":24522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24522},{"type":"v1","addr":"172.21.15.38:6807","nonce":24522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24522},{"type":"v1","addr":"172.21.15.38:6817","nonce":24522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24522},{"type":"v1","addr":"172.21.15.38:6812","nonce":24522}]},"public_addr":"172.21.15.38:6801/24522","cluster_addr":"172.21.15.38:6807/24522","heartbeat_back_addr":"172.21.15.38:6817/24522","heartbeat_front_addr":"172.21.15.38:6812/24522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.171690+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.165049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.245376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.051689+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-29T18:04:03.180 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 10 mgrc shutdown 2022-11-29T18:04:03.181 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6b88084de0 msgr2=0x7f6b880872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.181 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6b88084de0 0x7f6b880872a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f6ba0001560 tx=0x7f6ba0007f10).stop 2022-11-29T18:04:03.181 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6ba80b0fb0 msgr2=0x7f6ba80acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.182 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6ba80b0fb0 0x7f6ba80acac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f6b9c00c4f0 tx=0x7f6b9c007e00).stop 2022-11-29T18:04:03.182 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 shutdown_connections 2022-11-29T18:04:03.182 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6b88084de0 0x7f6b880872a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.182 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6ba8144680 0x7f6ba80acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.183 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6ba80b0fb0 0x7f6ba80acac0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.183 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 --2- 172.21.15.35:0/1024644914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ba80a2920 0x7f6ba8067e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.183 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 >> 172.21.15.35:0/1024644914 conn(0x7f6ba8069bf0 msgr2=0x7f6ba809a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:03.183 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 shutdown_connections 2022-11-29T18:04:03.184 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.178+0000 7f6baddec700 1 -- 172.21.15.35:0/1024644914 wait complete. 2022-11-29T18:04:03.192 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2022-11-29T18:04:01.750436+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'device_health_metrics', 'create_time': '2022-11-29T18:04:01.898453+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}}] 2022-11-29T18:04:03.194 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-29T18:04:03.302 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f7943ed1700 1 Processor -- start 2022-11-29T18:04:03.303 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f7943ed1700 1 -- start start 2022-11-29T18:04:03.306 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f7943ed1700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f793c143d50 0x7f793c146140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f7943ed1700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c09a9d0 con 0x7f793c0a2790 2022-11-29T18:04:03.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f7943ed1700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c13e8d0 con 0x7f793c13e4f0 2022-11-29T18:04:03.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f7943ed1700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c09b8e0 con 0x7f793c143d50 2022-11-29T18:04:03.307 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f794146c700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f793c0a2790 0x7f793c0b1170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50300/0 (socket says 172.21.15.35:50300) 2022-11-29T18:04:03.308 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f794146c700 1 -- 172.21.15.35:0/2089740094 learned_addr learned my addr 172.21.15.35:0/2089740094 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:03.308 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f794246e700 1 --2- 172.21.15.35:0/2089740094 >> v2:172.21.15.38:3300/0 conn(0x7f793c143d50 0x7f793c146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.308 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f7940c6b700 1 -- 172.21.15.35:0/2089740094 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829456127 0 0) 0x7f793c09a9d0 con 0x7f793c0a2790 2022-11-29T18:04:03.308 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f7940c6b700 1 -- 172.21.15.35:0/2089740094 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79280033f0 con 0x7f793c0a2790 2022-11-29T18:04:03.309 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f794246e700 1 -- 172.21.15.35:0/2089740094 >> v1:172.21.15.38:6789/0 conn(0x7f793c13e4f0 legacy=0x7f793c09bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.309 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f794246e700 1 -- 172.21.15.35:0/2089740094 >> v1:172.21.15.35:6789/0 conn(0x7f793c0a2790 legacy=0x7f793c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.309 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f794246e700 1 -- 172.21.15.35:0/2089740094 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c0a2b70 con 0x7f793c143d50 2022-11-29T18:04:03.309 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.302+0000 7f794246e700 1 --2- 172.21.15.35:0/2089740094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 0x7f793c146140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7934009760 tx=0x7f7934009060).ready entity=mon.1 client_cookie=9251533c691c669e server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.309 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7940c6b700 1 -- 172.21.15.35:0/2089740094 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793400b590 con 0x7f793c143d50 2022-11-29T18:04:03.310 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7940c6b700 1 -- 172.21.15.35:0/2089740094 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793400b6f0 con 0x7f793c143d50 2022-11-29T18:04:03.310 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7940c6b700 1 -- 172.21.15.35:0/2089740094 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793400ba70 con 0x7f793c143d50 2022-11-29T18:04:03.310 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 -- 172.21.15.35:0/2089740094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 msgr2=0x7f793c146140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.310 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 --2- 172.21.15.35:0/2089740094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 0x7f793c146140 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7934009760 tx=0x7f7934009060).stop 2022-11-29T18:04:03.311 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 -- 172.21.15.35:0/2089740094 shutdown_connections 2022-11-29T18:04:03.311 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 --2- 172.21.15.35:0/2089740094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 0x7f793c146140 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.311 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 -- 172.21.15.35:0/2089740094 >> 172.21.15.35:0/2089740094 conn(0x7f793c069bb0 msgr2=0x7f793c0b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:03.311 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 -- 172.21.15.35:0/2089740094 shutdown_connections 2022-11-29T18:04:03.312 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 -- 172.21.15.35:0/2089740094 wait complete. 2022-11-29T18:04:03.316 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 Processor -- start 2022-11-29T18:04:03.316 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.306+0000 7f7943ed1700 1 -- start start 2022-11-29T18:04:03.316 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f793c0a2790 0x7f793c130450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.316 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f793c13e4f0 0x7f793c096bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.317 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 0x7f793c096290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.317 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c139c10 con 0x7f793c13e4f0 2022-11-29T18:04:03.317 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c130a00 con 0x7f793c143d50 2022-11-29T18:04:03.317 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c130cc0 con 0x7f793c0a2790 2022-11-29T18:04:03.318 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f794146c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f793c13e4f0 0x7f793c096bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.318 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f793c0a2790 0x7f793c130450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.318 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f793c0a2790 0x7f793c130450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57730/0 (socket says 172.21.15.35:57730) 2022-11-29T18:04:03.318 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 -- 172.21.15.35:0/3739868056 learned_addr learned my addr 172.21.15.35:0/3739868056 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:03.318 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f794246e700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 0x7f793c096290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.319 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 -- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 msgr2=0x7f793c096290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.319 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 0x7f793c096290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.319 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 -- 172.21.15.35:0/3739868056 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f793c13e4f0 msgr2=0x7f793c096bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.319 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f793c13e4f0 0x7f793c096bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.320 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 -- 172.21.15.35:0/3739868056 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340092a0 con 0x7f793c0a2790 2022-11-29T18:04:03.320 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f794146c700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f793c13e4f0 0x7f793c096bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-29T18:04:03.320 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7941c6d700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f793c0a2790 0x7f793c130450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f792c00c840 tx=0x7f792c001660).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.320 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7939ffb700 1 -- 172.21.15.35:0/3739868056 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c006a20 con 0x7f793c0a2790 2022-11-29T18:04:03.321 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7939ffb700 1 -- 172.21.15.35:0/3739868056 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c006b80 con 0x7f793c0a2790 2022-11-29T18:04:03.321 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c1317e0 con 0x7f793c0a2790 2022-11-29T18:04:03.321 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7939ffb700 1 -- 172.21.15.35:0/3739868056 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c007020 con 0x7f793c0a2790 2022-11-29T18:04:03.321 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c09dbd0 con 0x7f793c0a2790 2022-11-29T18:04:03.321 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.310+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c06b650 con 0x7f793c0a2790 2022-11-29T18:04:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f7939ffb700 1 -- 172.21.15.35:0/3739868056 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f792c010070 con 0x7f793c0a2790 2022-11-29T18:04:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f7939ffb700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f7939ffb700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:03.322 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f7939ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:03.323 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f7939ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:03.323 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f7939ffb700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f791c084de0 0x7f791c0872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.323 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f7939ffb700 1 -- 172.21.15.35:0/3739868056 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 3517+0+0 (secure 0 0 0) 0x7f792c0a6aa0 con 0x7f793c0a2790 2022-11-29T18:04:03.323 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f794146c700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f791c084de0 0x7f791c0872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.323 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.314+0000 7f794146c700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f791c084de0 0x7f791c0872a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f793c1327d0 tx=0x7f7930009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.324 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.318+0000 7f7939ffb700 1 -- 172.21.15.35:0/3739868056 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f792c047980 con 0x7f793c0a2790 2022-11-29T18:04:03.474 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.474+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f793c13e8d0 con 0x7f793c0a2790 2022-11-29T18:04:03.476 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.474+0000 7f7939ffb700 1 -- 172.21.15.35:0/3739868056 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v11) v1 ==== 92+0+10 (secure 0 0 0) 0x7f793c13e8d0 con 0x7f793c0a2790 2022-11-29T18:04:03.476 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 8 2022-11-29T18:04:03.481 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 10 mgrc shutdown 2022-11-29T18:04:03.481 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f791c084de0 msgr2=0x7f791c0872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.482 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f791c084de0 0x7f791c0872a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f793c1327d0 tx=0x7f7930009040).stop 2022-11-29T18:04:03.482 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f793c0a2790 msgr2=0x7f793c130450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.482 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f793c0a2790 0x7f793c130450 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f792c00c840 tx=0x7f792c001660).stop 2022-11-29T18:04:03.482 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 shutdown_connections 2022-11-29T18:04:03.482 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f791c084de0 0x7f791c0872a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.483 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f793c143d50 0x7f793c096290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.483 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f793c13e4f0 0x7f793c096bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.483 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 --2- 172.21.15.35:0/3739868056 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f793c0a2790 0x7f793c130450 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.483 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 >> 172.21.15.35:0/3739868056 conn(0x7f793c069bb0 msgr2=0x7f793c0af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:03.484 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 shutdown_connections 2022-11-29T18:04:03.484 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.478+0000 7f7943ed1700 1 -- 172.21.15.35:0/3739868056 wait complete. 2022-11-29T18:04:03.497 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-11-29T18:04:03.607 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548f07d700 1 Processor -- start 2022-11-29T18:04:03.608 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548f07d700 1 -- start start 2022-11-29T18:04:03.611 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548f07d700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f548809c970 0x7f548809cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.612 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548f07d700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5488099720 con 0x7f5488063880 2022-11-29T18:04:03.612 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548f07d700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548809e510 con 0x7f5488147140 2022-11-29T18:04:03.612 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548f07d700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548813e690 con 0x7f548809c970 2022-11-29T18:04:03.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d87a700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f548809c970 0x7f548809cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d87a700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f548809c970 0x7f548809cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43776/0 (socket says 172.21.15.35:43776) 2022-11-29T18:04:03.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d87a700 1 -- 172.21.15.35:0/1126560759 learned_addr learned my addr 172.21.15.35:0/1126560759 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:03.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548e07b700 1 --1- 172.21.15.35:0/1126560759 >> v1:172.21.15.35:6789/0 conn(0x7f5488063880 0x7f548813df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50304/0 (socket says 172.21.15.35:50304) 2022-11-29T18:04:03.613 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d87a700 1 -- 172.21.15.35:0/1126560759 >> v1:172.21.15.38:6789/0 conn(0x7f5488147140 legacy=0x7f5488149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d079700 1 -- 172.21.15.35:0/1126560759 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684470949 0 0) 0x7f548809e510 con 0x7f5488147140 2022-11-29T18:04:03.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d87a700 1 -- 172.21.15.35:0/1126560759 >> v1:172.21.15.35:6789/0 conn(0x7f5488063880 legacy=0x7f548813df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d87a700 1 -- 172.21.15.35:0/1126560759 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548809d340 con 0x7f548809c970 2022-11-29T18:04:03.614 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d87a700 1 --2- 172.21.15.35:0/1126560759 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f548809c970 0x7f548809cd90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f548809d530 tx=0x7f5478009840).ready entity=mon.0 client_cookie=15953536175c5b66 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.615 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d079700 1 -- 172.21.15.35:0/1126560759 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547800e070 con 0x7f548809c970 2022-11-29T18:04:03.615 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d079700 1 -- 172.21.15.35:0/1126560759 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547800b950 con 0x7f548809c970 2022-11-29T18:04:03.615 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.606+0000 7f548d079700 1 -- 172.21.15.35:0/1126560759 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547800bcd0 con 0x7f548809c970 2022-11-29T18:04:03.615 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- 172.21.15.35:0/1126560759 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f548809c970 msgr2=0x7f548809cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.616 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 --2- 172.21.15.35:0/1126560759 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f548809c970 0x7f548809cd90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f548809d530 tx=0x7f5478009840).stop 2022-11-29T18:04:03.617 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- 172.21.15.35:0/1126560759 shutdown_connections 2022-11-29T18:04:03.617 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 --2- 172.21.15.35:0/1126560759 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f548809c970 0x7f548809cd90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.617 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- 172.21.15.35:0/1126560759 >> 172.21.15.35:0/1126560759 conn(0x7f54880acf70 msgr2=0x7f5488062fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:03.617 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- 172.21.15.35:0/1126560759 shutdown_connections 2022-11-29T18:04:03.618 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- 172.21.15.35:0/1126560759 wait complete. 2022-11-29T18:04:03.618 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 Processor -- start 2022-11-29T18:04:03.618 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- start start 2022-11-29T18:04:03.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5488063880 0x7f548813c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f548809c970 0x7f5488144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5488147140 0x7f548813f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.622 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488144db0 con 0x7f5488147140 2022-11-29T18:04:03.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488139bd0 con 0x7f5488063880 2022-11-29T18:04:03.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548f07d700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5488140e20 con 0x7f548809c970 2022-11-29T18:04:03.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.610+0000 7f548e87c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5488147140 0x7f548813f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.623 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e87c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5488147140 0x7f548813f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43780/0 (socket says 172.21.15.35:43780) 2022-11-29T18:04:03.624 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548d87a700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f548809c970 0x7f5488144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.624 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e87c700 1 -- 172.21.15.35:0/3694580176 learned_addr learned my addr 172.21.15.35:0/3694580176 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:03.624 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e07b700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5488063880 0x7f548813c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.624 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e87c700 1 -- 172.21.15.35:0/3694580176 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f548809c970 msgr2=0x7f5488144370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.625 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e87c700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f548809c970 0x7f5488144370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.625 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e87c700 1 -- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5488063880 msgr2=0x7f548813c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.625 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e87c700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5488063880 0x7f548813c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.626 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e87c700 1 -- 172.21.15.35:0/3694580176 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5478009ae0 con 0x7f5488147140 2022-11-29T18:04:03.626 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548e87c700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5488147140 0x7f548813f860 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f548000c490 tx=0x7f5480007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.626 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f54867fc700 1 -- 172.21.15.35:0/3694580176 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480012400 con 0x7f5488147140 2022-11-29T18:04:03.626 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f54867fc700 1 -- 172.21.15.35:0/3694580176 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480012560 con 0x7f5488147140 2022-11-29T18:04:03.626 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548813cc60 con 0x7f5488147140 2022-11-29T18:04:03.627 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f54867fc700 1 -- 172.21.15.35:0/3694580176 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480012900 con 0x7f5488147140 2022-11-29T18:04:03.627 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548813bd20 con 0x7f5488147140 2022-11-29T18:04:03.627 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.614+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5488139890 con 0x7f5488147140 2022-11-29T18:04:03.627 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f54867fc700 1 -- 172.21.15.35:0/3694580176 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f5480019030 con 0x7f5488147140 2022-11-29T18:04:03.628 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f54867fc700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:03.628 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f54867fc700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:03.628 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f54867fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:03.628 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f54867fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:03.629 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f54867fc700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5468084de0 0x7f54680872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.629 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f54867fc700 1 -- 172.21.15.35:0/3694580176 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 3517+0+0 (secure 0 0 0) 0x7f54800a6710 con 0x7f5488147140 2022-11-29T18:04:03.629 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f548e07b700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5468084de0 0x7f54680872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.629 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.618+0000 7f548e07b700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5468084de0 0x7f54680872a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f547c004780 tx=0x7f547c007cb0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.630 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.626+0000 7f54867fc700 1 -- 172.21.15.35:0/3694580176 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f548007abe0 con 0x7f5488147140 2022-11-29T18:04:03.779 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.778+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f548809d600 con 0x7f5488147140 2022-11-29T18:04:03.780 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.778+0000 7f54867fc700 1 -- 172.21.15.35:0/3694580176 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v11) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5480017030 con 0x7f5488147140 2022-11-29T18:04:03.781 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 1 2022-11-29T18:04:03.785 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 10 mgrc shutdown 2022-11-29T18:04:03.785 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5468084de0 msgr2=0x7f54680872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.785 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5468084de0 0x7f54680872a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f547c004780 tx=0x7f547c007cb0).stop 2022-11-29T18:04:03.785 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5488147140 msgr2=0x7f548813f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.786 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5488147140 0x7f548813f860 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f548000c490 tx=0x7f5480007d80).stop 2022-11-29T18:04:03.786 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 shutdown_connections 2022-11-29T18:04:03.786 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5468084de0 0x7f54680872a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.786 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5488147140 0x7f548813f860 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.786 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f548809c970 0x7f5488144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.787 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 --2- 172.21.15.35:0/3694580176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5488063880 0x7f548813c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.787 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 >> 172.21.15.35:0/3694580176 conn(0x7f54880acf70 msgr2=0x7f5488061f70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:03.787 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 shutdown_connections 2022-11-29T18:04:03.787 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.782+0000 7f548f07d700 1 -- 172.21.15.35:0/3694580176 wait complete. 2022-11-29T18:04:03.802 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-29T18:04:03.908 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.906+0000 7f2565776700 1 Processor -- start 2022-11-29T18:04:03.909 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.906+0000 7f2565776700 1 -- start start 2022-11-29T18:04:03.911 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600a2920 0x7f2560145c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.911 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256009c680 con 0x7f25600b0fb0 2022-11-29T18:04:03.912 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256013b320 con 0x7f25600b1390 2022-11-29T18:04:03.912 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600a2d90 con 0x7f25600a2920 2022-11-29T18:04:03.912 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2564f75700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600a2920 0x7f2560145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.913 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2564f75700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600a2920 0x7f2560145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57752/0 (socket says 172.21.15.35:57752) 2022-11-29T18:04:03.913 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2564f75700 1 -- 172.21.15.35:0/1130864104 learned_addr learned my addr 172.21.15.35:0/1130864104 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:03.913 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f255ffff700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f25600b0fb0 0x7f256013ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50314/0 (socket says 172.21.15.35:50314) 2022-11-29T18:04:03.913 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2564f75700 1 -- 172.21.15.35:0/1130864104 >> v1:172.21.15.38:6789/0 conn(0x7f25600b1390 legacy=0x7f256009bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.914 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f255f7fe700 1 -- 172.21.15.35:0/1130864104 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923565795 0 0) 0x7f256009c680 con 0x7f25600b0fb0 2022-11-29T18:04:03.914 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2564f75700 1 -- 172.21.15.35:0/1130864104 >> v1:172.21.15.35:6789/0 conn(0x7f25600b0fb0 legacy=0x7f256013ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.914 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2564f75700 1 -- 172.21.15.35:0/1130864104 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256009ae60 con 0x7f25600a2920 2022-11-29T18:04:03.914 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2564f75700 1 --2- 172.21.15.35:0/1130864104 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600a2920 0x7f2560145c20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2558009f50 tx=0x7f2558009840).ready entity=mon.2 client_cookie=2b537162b924ddb6 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.915 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f255f7fe700 1 -- 172.21.15.35:0/1130864104 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800e070 con 0x7f25600a2920 2022-11-29T18:04:03.915 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f255f7fe700 1 -- 172.21.15.35:0/1130864104 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255800bab0 con 0x7f25600a2920 2022-11-29T18:04:03.915 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f255f7fe700 1 -- 172.21.15.35:0/1130864104 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800be30 con 0x7f25600a2920 2022-11-29T18:04:03.915 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- 172.21.15.35:0/1130864104 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600a2920 msgr2=0x7f2560145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.916 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 --2- 172.21.15.35:0/1130864104 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600a2920 0x7f2560145c20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2558009f50 tx=0x7f2558009840).stop 2022-11-29T18:04:03.916 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- 172.21.15.35:0/1130864104 shutdown_connections 2022-11-29T18:04:03.916 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 --2- 172.21.15.35:0/1130864104 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600a2920 0x7f2560145c20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.916 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- 172.21.15.35:0/1130864104 >> 172.21.15.35:0/1130864104 conn(0x7f2560069bf0 msgr2=0x7f25600b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:03.917 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- 172.21.15.35:0/1130864104 shutdown_connections 2022-11-29T18:04:03.917 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- 172.21.15.35:0/1130864104 wait complete. 2022-11-29T18:04:03.921 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 Processor -- start 2022-11-29T18:04:03.921 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.910+0000 7f2565776700 1 -- start start 2022-11-29T18:04:03.921 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600a2920 0x7f25600a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.922 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600b0fb0 0x7f2560137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.922 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f25600a6160 0x7f25600a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.922 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600a7e60 con 0x7f25600a6160 2022-11-29T18:04:03.922 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600a4920 con 0x7f25600a2920 2022-11-29T18:04:03.923 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2560137880 con 0x7f25600b0fb0 2022-11-29T18:04:03.923 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f25600a6160 0x7f25600a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.923 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f25577fe700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600b0fb0 0x7f2560137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.923 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f25600a6160 0x7f25600a8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43796/0 (socket says 172.21.15.35:43796) 2022-11-29T18:04:03.923 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 -- 172.21.15.35:0/4107775098 learned_addr learned my addr 172.21.15.35:0/4107775098 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:03.924 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f255ffff700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600a2920 0x7f25600a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.924 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 -- 172.21.15.35:0/4107775098 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600b0fb0 msgr2=0x7f2560137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.924 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600b0fb0 0x7f2560137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.924 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 -- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600a2920 msgr2=0x7f25600a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:03.925 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600a2920 0x7f25600a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:03.925 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 -- 172.21.15.35:0/4107775098 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558009ae0 con 0x7f25600a6160 2022-11-29T18:04:03.925 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2564f75700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f25600a6160 0x7f25600a8ac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f255800b770 tx=0x7f256009aff0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.925 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f255cff9700 1 -- 172.21.15.35:0/4107775098 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580092e0 con 0x7f25600a6160 2022-11-29T18:04:03.925 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f255cff9700 1 -- 172.21.15.35:0/4107775098 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255801e470 con 0x7f25600a6160 2022-11-29T18:04:03.926 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25600a8fb0 con 0x7f25600a6160 2022-11-29T18:04:03.926 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f255cff9700 1 -- 172.21.15.35:0/4107775098 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255801e810 con 0x7f25600a6160 2022-11-29T18:04:03.926 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25600a94a0 con 0x7f25600a6160 2022-11-29T18:04:03.926 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.914+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2560046830 con 0x7f25600a6160 2022-11-29T18:04:03.927 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255cff9700 1 -- 172.21.15.35:0/4107775098 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f2558025030 con 0x7f25600a6160 2022-11-29T18:04:03.927 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255cff9700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:03.928 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255cff9700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:03.928 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255cff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:03.928 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255cff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:03.928 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255cff9700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f2544084de0 0x7f25440872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:03.929 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255cff9700 1 -- 172.21.15.35:0/4107775098 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 3541+0+0 (secure 0 0 0) 0x7f25580a8a30 con 0x7f25600a6160 2022-11-29T18:04:03.929 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255ffff700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f2544084de0 0x7f25440872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:03.929 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.918+0000 7f255ffff700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f2544084de0 0x7f25440872a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f25500011e0 tx=0x7f2550007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:03.930 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:03.922+0000 7f255cff9700 1 -- 172.21.15.35:0/4107775098 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f255807cee0 con 0x7f25600a6160 2022-11-29T18:04:04.110 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.110+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f256009c8d0 con 0x7f25600a6160 2022-11-29T18:04:04.111 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.110+0000 7f255cff9700 1 -- 172.21.15.35:0/4107775098 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2765 (secure 0 0 0) 0x7f2558023030 con 0x7f25600a6160 2022-11-29T18:04:04.112 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:04:04.112 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.35:6811/336810684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":336810684},{"type":"v1","addr":"172.21.15.35:6811","nonce":336810684}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4202,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/2614917468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2614917468},{"type":"v1","addr":"172.21.15.38:6827","nonce":2614917468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4215,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.35:6809/2456090531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":2456090531},{"type":"v1","addr":"172.21.15.35:6809","nonce":2456090531}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2022-11-29T18:04:04.117 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 10 mgrc shutdown 2022-11-29T18:04:04.118 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f2544084de0 msgr2=0x7f25440872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.119 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f2544084de0 0x7f25440872a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f25500011e0 tx=0x7f2550007bd0).stop 2022-11-29T18:04:04.119 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f25600a6160 msgr2=0x7f25600a8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.120 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f25600a6160 0x7f25600a8ac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f255800b770 tx=0x7f256009aff0).stop 2022-11-29T18:04:04.120 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 shutdown_connections 2022-11-29T18:04:04.120 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f2544084de0 0x7f25440872a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.121 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f25600a6160 0x7f25600a8ac0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.121 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f25600b0fb0 0x7f2560137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.122 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 --2- 172.21.15.35:0/4107775098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25600a2920 0x7f25600a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.122 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 >> 172.21.15.35:0/4107775098 conn(0x7f2560069bf0 msgr2=0x7f256009a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:04.123 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 shutdown_connections 2022-11-29T18:04:04.123 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.114+0000 7f2565776700 1 -- 172.21.15.35:0/4107775098 wait complete. 2022-11-29T18:04:04.124 INFO:teuthology.orchestra.run.smithi035.stderr:dumped fsmap epoch 3 2022-11-29T18:04:04.126 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-29T18:04:04.233 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.230+0000 7f6a18852700 1 Processor -- start 2022-11-29T18:04:04.234 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- start start 2022-11-29T18:04:04.237 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a14137da0 0x7f6a140adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.237 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a14068c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.238 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140696d0 con 0x7f6a14069140 2022-11-29T18:04:04.238 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a14138180 con 0x7f6a14137da0 2022-11-29T18:04:04.238 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a141370f0 con 0x7f6a14069b90 2022-11-29T18:04:04.239 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a1259c700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a14068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.239 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a1259c700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a14068c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57766/0 (socket says 172.21.15.35:57766) 2022-11-29T18:04:04.239 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a1259c700 1 -- 172.21.15.35:0/3056664983 learned_addr learned my addr 172.21.15.35:0/3056664983 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:04.239 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a12d9d700 1 --2- 172.21.15.35:0/3056664983 >> v2:172.21.15.38:3300/0 conn(0x7f6a14137da0 0x7f6a140adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.240 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a1259c700 1 -- 172.21.15.35:0/3056664983 >> v2:172.21.15.38:3300/0 conn(0x7f6a14137da0 msgr2=0x7f6a140adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:04.240 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a1259c700 1 --2- 172.21.15.35:0/3056664983 >> v2:172.21.15.38:3300/0 conn(0x7f6a14137da0 0x7f6a140adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.240 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a1259c700 1 -- 172.21.15.35:0/3056664983 >> v1:172.21.15.38:6789/0 conn(0x7f6a14069140 legacy=0x7f6a1413a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.241 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a1259c700 1 -- 172.21.15.35:0/3056664983 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a14137350 con 0x7f6a14069b90 2022-11-29T18:04:04.241 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a1259c700 1 --2- 172.21.15.35:0/3056664983 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a14068c50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f69fc006fd0 tx=0x7f69fc0090f0).ready entity=mon.2 client_cookie=e3cc8987038391d3 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.241 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a11d9b700 1 -- 172.21.15.35:0/3056664983 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00b610 con 0x7f6a14069b90 2022-11-29T18:04:04.241 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a11d9b700 1 -- 172.21.15.35:0/3056664983 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc00b770 con 0x7f6a14069b90 2022-11-29T18:04:04.242 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a11d9b700 1 -- 172.21.15.35:0/3056664983 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00baf0 con 0x7f6a14069b90 2022-11-29T18:04:04.242 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- 172.21.15.35:0/3056664983 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 msgr2=0x7f6a14068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.242 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 --2- 172.21.15.35:0/3056664983 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a14068c50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f69fc006fd0 tx=0x7f69fc0090f0).stop 2022-11-29T18:04:04.242 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- 172.21.15.35:0/3056664983 shutdown_connections 2022-11-29T18:04:04.243 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 --2- 172.21.15.35:0/3056664983 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a14068c50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.243 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 --2- 172.21.15.35:0/3056664983 >> v2:172.21.15.38:3300/0 conn(0x7f6a14137da0 0x7f6a140adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.243 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- 172.21.15.35:0/3056664983 >> 172.21.15.35:0/3056664983 conn(0x7f6a140635d0 msgr2=0x7f6a140ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:04.244 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- 172.21.15.35:0/3056664983 shutdown_connections 2022-11-29T18:04:04.244 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.234+0000 7f6a18852700 1 -- 172.21.15.35:0/3056664983 wait complete. 2022-11-29T18:04:04.244 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 Processor -- start 2022-11-29T18:04:04.244 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 -- start start 2022-11-29T18:04:04.248 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14069140 0x7f6a14133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.248 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a140a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.248 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6a14137da0 0x7f6a14132840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.249 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1414a330 con 0x7f6a14137da0 2022-11-29T18:04:04.249 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14139560 con 0x7f6a14069140 2022-11-29T18:04:04.249 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14139800 con 0x7f6a14069b90 2022-11-29T18:04:04.249 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1259c700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a140a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6a14137da0 0x7f6a14132840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a12d9d700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14069140 0x7f6a14133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6a14137da0 0x7f6a14132840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43798/0 (socket says 172.21.15.35:43798) 2022-11-29T18:04:04.250 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 -- 172.21.15.35:0/3744172649 learned_addr learned my addr 172.21.15.35:0/3744172649 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:04.251 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 -- 172.21.15.35:0/3744172649 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 msgr2=0x7f6a140a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.251 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a140a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.251 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 -- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14069140 msgr2=0x7f6a14133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.251 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14069140 0x7f6a14133710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.252 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 -- 172.21.15.35:0/3744172649 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc009370 con 0x7f6a14137da0 2022-11-29T18:04:04.252 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a1359e700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6a14137da0 0x7f6a14132840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f6a0c002230 tx=0x7f6a0c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.252 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a0affd700 1 -- 172.21.15.35:0/3744172649 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00e4a0 con 0x7f6a14137da0 2022-11-29T18:04:04.252 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a0affd700 1 -- 172.21.15.35:0/3744172649 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c00e600 con 0x7f6a14137da0 2022-11-29T18:04:04.253 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a0affd700 1 -- 172.21.15.35:0/3744172649 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00e980 con 0x7f6a14137da0 2022-11-29T18:04:04.253 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a14139aa0 con 0x7f6a14137da0 2022-11-29T18:04:04.253 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a140a4430 con 0x7f6a14137da0 2022-11-29T18:04:04.253 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.238+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a14046830 con 0x7f6a14137da0 2022-11-29T18:04:04.254 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a0affd700 1 -- 172.21.15.35:0/3744172649 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f6a0c019030 con 0x7f6a14137da0 2022-11-29T18:04:04.254 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a0affd700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:04.254 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a0affd700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:04.255 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a0affd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:04.255 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a0affd700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:04.255 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a0affd700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f69f8084de0 0x7f69f80872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.255 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a0affd700 1 -- 172.21.15.35:0/3744172649 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 3541+0+0 (secure 0 0 0) 0x7f6a0c0a6e60 con 0x7f6a14137da0 2022-11-29T18:04:04.256 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a12d9d700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f69f8084de0 0x7f69f80872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.256 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.242+0000 7f6a12d9d700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f69f8084de0 0x7f69f80872a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f6a04009b90 tx=0x7f6a04009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.256 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.246+0000 7f6a0affd700 1 -- 172.21.15.35:0/3744172649 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f6a0c07b310 con 0x7f6a14137da0 2022-11-29T18:04:04.402 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.402+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a1409fd60 con 0x7f6a14137da0 2022-11-29T18:04:04.403 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.402+0000 7f6a0affd700 1 -- 172.21.15.35:0/3744172649 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+9017 (secure 0 0 0) 0x7f6a0c017030 con 0x7f6a14137da0 2022-11-29T18:04:04.403 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:04:04.404 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":12,"fsid":"338b2b5b-2e59-4e89-9321-f2b909af04fd","created":"2022-11-29T18:03:52.771747+0000","modified":"2022-11-29T18:04:03.836592+0000","last_up_change":"2022-11-29T18:04:00.802110+0000","last_in_change":"2022-11-29T18:03:53.874582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"rbd","create_time":"2022-11-29T18:04:01.750436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"device_health_metrics","create_time":"2022-11-29T18:04:01.898453+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"228fad84-4a8f-44ce-ba6d-f0433a5a22ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":22147},{"type":"v1","addr":"172.21.15.35:6801","nonce":22147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":22147},{"type":"v1","addr":"172.21.15.35:6803","nonce":22147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":22147},{"type":"v1","addr":"172.21.15.35:6807","nonce":22147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":22147},{"type":"v1","addr":"172.21.15.35:6805","nonce":22147}]},"public_addr":"172.21.15.35:6801/22147","cluster_addr":"172.21.15.35:6803/22147","heartbeat_back_addr":"172.21.15.35:6807/22147","heartbeat_front_addr":"172.21.15.35:6805/22147","state":["exists","up"]},{"osd":1,"uuid":"d58467d0-703a-4da6-980d-30bf1051838b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24521},{"type":"v1","addr":"172.21.15.38:6803","nonce":24521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6805","nonce":24521},{"type":"v1","addr":"172.21.15.38:6806","nonce":24521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6811","nonce":24521},{"type":"v1","addr":"172.21.15.38:6815","nonce":24521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24521},{"type":"v1","addr":"172.21.15.38:6809","nonce":24521}]},"public_addr":"172.21.15.38:6803/24521","cluster_addr":"172.21.15.38:6806/24521","heartbeat_back_addr":"172.21.15.38:6815/24521","heartbeat_front_addr":"172.21.15.38:6809/24521","state":["exists","up"]},{"osd":2,"uuid":"34a91d7b-b17b-4400-b2f3-0de86be92ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6813","nonce":24523},{"type":"v1","addr":"172.21.15.38:6816","nonce":24523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24523},{"type":"v1","addr":"172.21.15.38:6819","nonce":24523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24523},{"type":"v1","addr":"172.21.15.38:6823","nonce":24523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24523},{"type":"v1","addr":"172.21.15.38:6821","nonce":24523}]},"public_addr":"172.21.15.38:6816/24523","cluster_addr":"172.21.15.38:6819/24523","heartbeat_back_addr":"172.21.15.38:6823/24523","heartbeat_front_addr":"172.21.15.38:6821/24523","state":["exists","up"]},{"osd":3,"uuid":"c7d3a292-4f7e-456d-ad04-0a67f33eb539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24522},{"type":"v1","addr":"172.21.15.38:6801","nonce":24522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24522},{"type":"v1","addr":"172.21.15.38:6807","nonce":24522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24522},{"type":"v1","addr":"172.21.15.38:6817","nonce":24522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24522},{"type":"v1","addr":"172.21.15.38:6812","nonce":24522}]},"public_addr":"172.21.15.38:6801/24522","cluster_addr":"172.21.15.38:6807/24522","heartbeat_back_addr":"172.21.15.38:6817/24522","heartbeat_front_addr":"172.21.15.38:6812/24522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.171690+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.165049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.245376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.051689+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-29T18:04:04.406 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 10 mgrc shutdown 2022-11-29T18:04:04.406 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f69f8084de0 msgr2=0x7f69f80872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.406 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f69f8084de0 0x7f69f80872a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f6a04009b90 tx=0x7f6a04009060).stop 2022-11-29T18:04:04.407 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6a14137da0 msgr2=0x7f6a14132840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.407 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6a14137da0 0x7f6a14132840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f6a0c002230 tx=0x7f6a0c0015e0).stop 2022-11-29T18:04:04.407 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 shutdown_connections 2022-11-29T18:04:04.407 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f69f8084de0 0x7f69f80872a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.408 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6a14137da0 0x7f6a14132840 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.408 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f6a14069b90 0x7f6a140a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.408 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 --2- 172.21.15.35:0/3744172649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14069140 0x7f6a14133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.408 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 >> 172.21.15.35:0/3744172649 conn(0x7f6a140635d0 msgr2=0x7f6a14134090 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:04.408 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 shutdown_connections 2022-11-29T18:04:04.409 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.406+0000 7f6a18852700 1 -- 172.21.15.35:0/3744172649 wait complete. 2022-11-29T18:04:04.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2022-11-29T18:04:01.750436+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': 'device_health_metrics', 'create_time': '2022-11-29T18:04:01.898453+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}] 2022-11-29T18:04:04.420 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-29T18:04:04.535 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.534+0000 7f5d32fe5700 1 Processor -- start 2022-11-29T18:04:04.536 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.534+0000 7f5d32fe5700 1 -- start start 2022-11-29T18:04:04.538 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.534+0000 7f5d32fe5700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5d2c13a450 0x7f5d2c13a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.538 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.534+0000 7f5d32fe5700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0953f0 con 0x7f5d2c05d820 2022-11-29T18:04:04.539 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.534+0000 7f5d32fe5700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0965c0 con 0x7f5d2c142cb0 2022-11-29T18:04:04.539 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.534+0000 7f5d32fe5700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c13b3a0 con 0x7f5d2c13a450 2022-11-29T18:04:04.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d31fe3700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f5d2c05d820 0x7f5d2c139a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50316/0 (socket says 172.21.15.35:50316) 2022-11-29T18:04:04.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d31fe3700 1 -- 172.21.15.35:0/3201725706 learned_addr learned my addr 172.21.15.35:0/3201725706 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:04.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d317e2700 1 --2- 172.21.15.35:0/3201725706 >> v2:172.21.15.38:3300/0 conn(0x7f5d2c13a450 0x7f5d2c13a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d30fe1700 1 -- 172.21.15.35:0/3201725706 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660114718 0 0) 0x7f5d2c0953f0 con 0x7f5d2c05d820 2022-11-29T18:04:04.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d30fe1700 1 -- 172.21.15.35:0/3201725706 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d140033f0 con 0x7f5d2c05d820 2022-11-29T18:04:04.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d317e2700 1 -- 172.21.15.35:0/3201725706 >> v1:172.21.15.38:6789/0 conn(0x7f5d2c142cb0 legacy=0x7f5d2c1450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.542 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d30fe1700 1 -- 172.21.15.35:0/3201725706 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233448937 0 0) 0x7f5d2c0965c0 con 0x7f5d2c142cb0 2022-11-29T18:04:04.542 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d317e2700 1 -- 172.21.15.35:0/3201725706 >> v1:172.21.15.35:6789/0 conn(0x7f5d2c05d820 legacy=0x7f5d2c139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.543 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d317e2700 1 -- 172.21.15.35:0/3201725706 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c094450 con 0x7f5d2c13a450 2022-11-29T18:04:04.543 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d317e2700 1 --2- 172.21.15.35:0/3201725706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c13a450 0x7f5d2c13a8e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f5d2c0945e0 tx=0x7f5d1c009060).ready entity=mon.1 client_cookie=df19454febc96e75 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.550 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d30fe1700 1 -- 172.21.15.35:0/3201725706 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c009120 con 0x7f5d2c13a450 2022-11-29T18:04:04.551 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d30fe1700 1 -- 172.21.15.35:0/3201725706 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c00f510 con 0x7f5d2c13a450 2022-11-29T18:04:04.551 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d30fe1700 1 -- 172.21.15.35:0/3201725706 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00f890 con 0x7f5d2c13a450 2022-11-29T18:04:04.552 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d32fe5700 1 -- 172.21.15.35:0/3201725706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c13a450 msgr2=0x7f5d2c13a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.552 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d32fe5700 1 --2- 172.21.15.35:0/3201725706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c13a450 0x7f5d2c13a8e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f5d2c0945e0 tx=0x7f5d1c009060).stop 2022-11-29T18:04:04.552 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d32fe5700 1 -- 172.21.15.35:0/3201725706 shutdown_connections 2022-11-29T18:04:04.553 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d32fe5700 1 --2- 172.21.15.35:0/3201725706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c13a450 0x7f5d2c13a8e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.553 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d32fe5700 1 -- 172.21.15.35:0/3201725706 >> 172.21.15.35:0/3201725706 conn(0x7f5d2c0624c0 msgr2=0x7f5d2c0acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:04.554 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d32fe5700 1 -- 172.21.15.35:0/3201725706 shutdown_connections 2022-11-29T18:04:04.554 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.538+0000 7f5d32fe5700 1 -- 172.21.15.35:0/3201725706 wait complete. 2022-11-29T18:04:04.555 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d32fe5700 1 Processor -- start 2022-11-29T18:04:04.555 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d32fe5700 1 -- start start 2022-11-29T18:04:04.555 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d32fe5700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c05d820 0x7f5d2c1368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.556 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d32fe5700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d2c13a450 0x7f5d2c068350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.556 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d32fe5700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d2c142cb0 0x7f5d2c09d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.556 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d32fe5700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c068fb0 con 0x7f5d2c13a450 2022-11-29T18:04:04.556 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d32fe5700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c099c70 con 0x7f5d2c05d820 2022-11-29T18:04:04.557 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d32fe5700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c09dc30 con 0x7f5d2c142cb0 2022-11-29T18:04:04.557 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d2c13a450 0x7f5d2c068350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.557 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d2c13a450 0x7f5d2c068350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43810/0 (socket says 172.21.15.35:43810) 2022-11-29T18:04:04.558 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 -- 172.21.15.35:0/2730767352 learned_addr learned my addr 172.21.15.35:0/2730767352 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:04.558 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d327e4700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d2c142cb0 0x7f5d2c09d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.558 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d31fe3700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c05d820 0x7f5d2c1368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.558 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 -- 172.21.15.35:0/2730767352 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d2c142cb0 msgr2=0x7f5d2c09d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.559 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d2c142cb0 0x7f5d2c09d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.559 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 -- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c05d820 msgr2=0x7f5d2c1368b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.559 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c05d820 0x7f5d2c1368b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.559 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 -- 172.21.15.35:0/2730767352 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c0092a0 con 0x7f5d2c13a450 2022-11-29T18:04:04.560 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d317e2700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d2c13a450 0x7f5d2c068350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5d1c009dd0 tx=0x7f5d1c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.560 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.542+0000 7f5d19ffb700 1 -- 172.21.15.35:0/2730767352 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00f890 con 0x7f5d2c13a450 2022-11-29T18:04:04.560 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 1 -- 172.21.15.35:0/2730767352 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c000c40 con 0x7f5d2c13a450 2022-11-29T18:04:04.560 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c09def0 con 0x7f5d2c13a450 2022-11-29T18:04:04.561 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 1 -- 172.21.15.35:0/2730767352 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00fe80 con 0x7f5d2c13a450 2022-11-29T18:04:04.561 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c09eab0 con 0x7f5d2c13a450 2022-11-29T18:04:04.561 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c099e70 con 0x7f5d2c13a450 2022-11-29T18:04:04.561 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 1 -- 172.21.15.35:0/2730767352 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f5d1c00f070 con 0x7f5d2c13a450 2022-11-29T18:04:04.561 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:04.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:04.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:04.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:04.562 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5d0c084e10 0x7f5d0c0872d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.563 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d19ffb700 1 -- 172.21.15.35:0/2730767352 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 3541+0+0 (secure 0 0 0) 0x7f5d1c0a8b30 con 0x7f5d2c13a450 2022-11-29T18:04:04.563 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.546+0000 7f5d31fe3700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5d0c084e10 0x7f5d0c0872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.563 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.550+0000 7f5d31fe3700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5d0c084e10 0x7f5d0c0872d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5d28005750 tx=0x7f5d28009040).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.564 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.558+0000 7f5d19ffb700 1 -- 172.21.15.35:0/2730767352 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f5d1c07cfe0 con 0x7f5d2c13a450 2022-11-29T18:04:04.713 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.710+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d2c13b0e0 con 0x7f5d2c13a450 2022-11-29T18:04:04.714 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d19ffb700 1 -- 172.21.15.35:0/2730767352 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v12) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d1c080600 con 0x7f5d2c13a450 2022-11-29T18:04:04.714 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 8 2022-11-29T18:04:04.717 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 10 mgrc shutdown 2022-11-29T18:04:04.717 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5d0c084e10 msgr2=0x7f5d0c0872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.717 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5d0c084e10 0x7f5d0c0872d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f5d28005750 tx=0x7f5d28009040).stop 2022-11-29T18:04:04.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d2c13a450 msgr2=0x7f5d2c068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d2c13a450 0x7f5d2c068350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f5d1c009dd0 tx=0x7f5d1c017000).stop 2022-11-29T18:04:04.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 shutdown_connections 2022-11-29T18:04:04.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f5d0c084e10 0x7f5d0c0872d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.719 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f5d2c142cb0 0x7f5d2c09d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.720 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f5d2c13a450 0x7f5d2c068350 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.720 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 --2- 172.21.15.35:0/2730767352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d2c05d820 0x7f5d2c1368b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.720 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 >> 172.21.15.35:0/2730767352 conn(0x7f5d2c0624c0 msgr2=0x7f5d2c0aac90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:04.720 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 shutdown_connections 2022-11-29T18:04:04.721 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.714+0000 7f5d32fe5700 1 -- 172.21.15.35:0/2730767352 wait complete. 2022-11-29T18:04:04.729 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2022-11-29T18:04:04.844 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.842+0000 7f42ef59e700 1 Processor -- start 2022-11-29T18:04:04.845 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.842+0000 7f42ef59e700 1 -- start start 2022-11-29T18:04:04.847 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f013e270 0x7f42f0067ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.847 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f00683d0 con 0x7f42f0064bd0 2022-11-29T18:04:04.848 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f013da20 con 0x7f42f00647f0 2022-11-29T18:04:04.848 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f013dce0 con 0x7f42f013e270 2022-11-29T18:04:04.848 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42eed9d700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f42f0064bd0 0x7f42f0144c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50332/0 (socket says 172.21.15.35:50332) 2022-11-29T18:04:04.849 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42eed9d700 1 -- 172.21.15.35:0/1517976469 learned_addr learned my addr 172.21.15.35:0/1517976469 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:04.849 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ee59c700 1 --2- 172.21.15.35:0/1517976469 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f013e270 0x7f42f0067ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.850 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ed59a700 1 -- 172.21.15.35:0/1517976469 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749910674 0 0) 0x7f42f00683d0 con 0x7f42f0064bd0 2022-11-29T18:04:04.850 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ed59a700 1 -- 172.21.15.35:0/1517976469 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d00033f0 con 0x7f42f0064bd0 2022-11-29T18:04:04.850 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ee59c700 1 -- 172.21.15.35:0/1517976469 >> v1:172.21.15.38:6789/0 conn(0x7f42f00647f0 legacy=0x7f42f0061f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.850 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ed59a700 1 -- 172.21.15.35:0/1517976469 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647620351 0 0) 0x7f42f013da20 con 0x7f42f00647f0 2022-11-29T18:04:04.851 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ee59c700 1 -- 172.21.15.35:0/1517976469 >> v1:172.21.15.35:6789/0 conn(0x7f42f0064bd0 legacy=0x7f42f0144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.852 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ee59c700 1 -- 172.21.15.35:0/1517976469 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f009e1c0 con 0x7f42f013e270 2022-11-29T18:04:04.852 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ee59c700 1 --2- 172.21.15.35:0/1517976469 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f013e270 0x7f42f0067ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f42d8009790 tx=0x7f42d8009060).ready entity=mon.2 client_cookie=422a622150788b01 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.852 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ed59a700 1 -- 172.21.15.35:0/1517976469 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d800b590 con 0x7f42f013e270 2022-11-29T18:04:04.853 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ed59a700 1 -- 172.21.15.35:0/1517976469 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d800b6f0 con 0x7f42f013e270 2022-11-29T18:04:04.853 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ed59a700 1 -- 172.21.15.35:0/1517976469 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d800ba70 con 0x7f42f013e270 2022-11-29T18:04:04.857 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 -- 172.21.15.35:0/1517976469 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f013e270 msgr2=0x7f42f0067ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.857 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 --2- 172.21.15.35:0/1517976469 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f013e270 0x7f42f0067ee0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f42d8009790 tx=0x7f42d8009060).stop 2022-11-29T18:04:04.858 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 -- 172.21.15.35:0/1517976469 shutdown_connections 2022-11-29T18:04:04.858 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 --2- 172.21.15.35:0/1517976469 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f013e270 0x7f42f0067ee0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.858 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 -- 172.21.15.35:0/1517976469 >> 172.21.15.35:0/1517976469 conn(0x7f42f00ae040 msgr2=0x7f42f00640e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:04.858 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 -- 172.21.15.35:0/1517976469 shutdown_connections 2022-11-29T18:04:04.859 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.846+0000 7f42ef59e700 1 -- 172.21.15.35:0/1517976469 wait complete. 2022-11-29T18:04:04.859 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ef59e700 1 Processor -- start 2022-11-29T18:04:04.859 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ef59e700 1 -- start start 2022-11-29T18:04:04.860 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ef59e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f00647f0 0x7f42f01375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.860 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ef59e700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f42f013e270 0x7f42f014a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.860 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ef59e700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f014a9e0 0x7f42f009c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.861 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ef59e700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f0137ae0 con 0x7f42f013e270 2022-11-29T18:04:04.861 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ef59e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00971a0 con 0x7f42f00647f0 2022-11-29T18:04:04.862 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ef59e700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00973a0 con 0x7f42f014a9e0 2022-11-29T18:04:04.862 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f42f013e270 0x7f42f014a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.862 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f42f013e270 0x7f42f014a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43814/0 (socket says 172.21.15.35:43814) 2022-11-29T18:04:04.862 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 -- 172.21.15.35:0/2454092705 learned_addr learned my addr 172.21.15.35:0/2454092705 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:04.863 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42ee59c700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f00647f0 0x7f42f01375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.863 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42eed9d700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f014a9e0 0x7f42f009c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.863 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 -- 172.21.15.35:0/2454092705 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f014a9e0 msgr2=0x7f42f009c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.863 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f014a9e0 0x7f42f009c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.864 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 -- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f00647f0 msgr2=0x7f42f01375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:04.864 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f00647f0 0x7f42f01375f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:04.864 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 -- 172.21.15.35:0/2454092705 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80092a0 con 0x7f42f013e270 2022-11-29T18:04:04.864 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.850+0000 7f42edd9b700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f42f013e270 0x7f42f014a4f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f42e0001eb0 tx=0x7f42e0007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.865 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.858+0000 7f42e67fc700 1 -- 172.21.15.35:0/2454092705 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0012400 con 0x7f42f013e270 2022-11-29T18:04:04.865 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.858+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f014a050 con 0x7f42f013e270 2022-11-29T18:04:04.865 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.858+0000 7f42e67fc700 1 -- 172.21.15.35:0/2454092705 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e00125a0 con 0x7f42f013e270 2022-11-29T18:04:04.866 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.858+0000 7f42e67fc700 1 -- 172.21.15.35:0/2454092705 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0012960 con 0x7f42f013e270 2022-11-29T18:04:04.866 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.858+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f0098740 con 0x7f42f013e270 2022-11-29T18:04:04.868 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.858+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f014a210 con 0x7f42f013e270 2022-11-29T18:04:04.868 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.858+0000 7f42e67fc700 1 -- 172.21.15.35:0/2454092705 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f42e0019030 con 0x7f42f013e270 2022-11-29T18:04:04.868 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.858+0000 7f42e67fc700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:04.868 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.862+0000 7f42e67fc700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:04.869 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.862+0000 7f42e67fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:04.869 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.862+0000 7f42e67fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:04.869 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.862+0000 7f42e67fc700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f42d4084de0 0x7f42d40872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:04.870 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.862+0000 7f42e67fc700 1 -- 172.21.15.35:0/2454092705 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3541+0+0 (secure 0 0 0) 0x7f42e00a6680 con 0x7f42f013e270 2022-11-29T18:04:04.870 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.862+0000 7f42ee59c700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f42d4084de0 0x7f42d40872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:04.870 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.862+0000 7f42ee59c700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f42d4084de0 0x7f42d40872a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f42d80096e0 tx=0x7f42d8017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:04.871 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:04.866+0000 7f42e67fc700 1 -- 172.21.15.35:0/2454092705 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f42f014a210 con 0x7f42f013e270 2022-11-29T18:04:05.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.022+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f42f0046830 con 0x7f42f013e270 2022-11-29T18:04:05.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.022+0000 7f42e67fc700 1 -- 172.21.15.35:0/2454092705 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v13) v1 ==== 110+0+10 (secure 0 0 0) 0x7f42e0017030 con 0x7f42f013e270 2022-11-29T18:04:05.025 INFO:teuthology.orchestra.run.smithi035.stdout:pg_num: 1 2022-11-29T18:04:05.027 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 10 mgrc shutdown 2022-11-29T18:04:05.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f42d4084de0 msgr2=0x7f42d40872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f42d4084de0 0x7f42d40872a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f42d80096e0 tx=0x7f42d8017000).stop 2022-11-29T18:04:05.028 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f42f013e270 msgr2=0x7f42f014a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.029 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f42f013e270 0x7f42f014a4f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f42e0001eb0 tx=0x7f42e0007c40).stop 2022-11-29T18:04:05.029 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 shutdown_connections 2022-11-29T18:04:05.030 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f42d4084de0 0x7f42d40872a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.030 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f42f014a9e0 0x7f42f009c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.031 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f42f013e270 0x7f42f014a4f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.031 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 --2- 172.21.15.35:0/2454092705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42f00647f0 0x7f42f01375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.032 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 >> 172.21.15.35:0/2454092705 conn(0x7f42f00ae040 msgr2=0x7f42f0061e20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:05.032 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 shutdown_connections 2022-11-29T18:04:05.033 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.026+0000 7f42ef59e700 1 -- 172.21.15.35:0/2454092705 wait complete. 2022-11-29T18:04:05.039 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-29T18:04:05.137 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.134+0000 7f8830c04700 1 Processor -- start 2022-11-29T18:04:05.137 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.134+0000 7f8830c04700 1 -- start start 2022-11-29T18:04:05.140 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0a0200 0x7f882c09b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.141 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882c12e950 con 0x7f882c0aea10 2022-11-29T18:04:05.141 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882c09bb00 con 0x7f882c12dd90 2022-11-29T18:04:05.141 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c13bee0 con 0x7f882c0a0200 2022-11-29T18:04:05.141 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882affd700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f882c0aea10 0x7f882c13b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50340/0 (socket says 172.21.15.35:50340) 2022-11-29T18:04:05.142 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882affd700 1 -- 172.21.15.35:0/3478575905 learned_addr learned my addr 172.21.15.35:0/3478575905 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:05.142 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882b7fe700 1 --2- 172.21.15.35:0/3478575905 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0a0200 0x7f882c09b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.143 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882a7fc700 1 -- 172.21.15.35:0/3478575905 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835366752 0 0) 0x7f882c12e950 con 0x7f882c0aea10 2022-11-29T18:04:05.143 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882a7fc700 1 -- 172.21.15.35:0/3478575905 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f880c0033f0 con 0x7f882c0aea10 2022-11-29T18:04:05.144 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882a7fc700 1 -- 172.21.15.35:0/3478575905 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610508201 0 0) 0x7f882c09bb00 con 0x7f882c12dd90 2022-11-29T18:04:05.144 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882b7fe700 1 -- 172.21.15.35:0/3478575905 >> v1:172.21.15.38:6789/0 conn(0x7f882c12dd90 legacy=0x7f882c12e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.144 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882b7fe700 1 -- 172.21.15.35:0/3478575905 >> v1:172.21.15.35:6789/0 conn(0x7f882c0aea10 legacy=0x7f882c13b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.145 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882b7fe700 1 -- 172.21.15.35:0/3478575905 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c13c1a0 con 0x7f882c0a0200 2022-11-29T18:04:05.145 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882b7fe700 1 --2- 172.21.15.35:0/3478575905 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0a0200 0x7f882c09b610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8814009790 tx=0x7f8814009060).ready entity=mon.2 client_cookie=1805b7ea5ec48394 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:05.146 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882a7fc700 1 -- 172.21.15.35:0/3478575905 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881400b590 con 0x7f882c0a0200 2022-11-29T18:04:05.146 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882a7fc700 1 -- 172.21.15.35:0/3478575905 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881400b6f0 con 0x7f882c0a0200 2022-11-29T18:04:05.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f882a7fc700 1 -- 172.21.15.35:0/3478575905 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881400ba70 con 0x7f882c0a0200 2022-11-29T18:04:05.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 -- 172.21.15.35:0/3478575905 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0a0200 msgr2=0x7f882c09b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 --2- 172.21.15.35:0/3478575905 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0a0200 0x7f882c09b610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f8814009790 tx=0x7f8814009060).stop 2022-11-29T18:04:05.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 -- 172.21.15.35:0/3478575905 shutdown_connections 2022-11-29T18:04:05.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 --2- 172.21.15.35:0/3478575905 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0a0200 0x7f882c09b610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 -- 172.21.15.35:0/3478575905 >> 172.21.15.35:0/3478575905 conn(0x7f882c0619f0 msgr2=0x7f882c061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:05.150 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 -- 172.21.15.35:0/3478575905 shutdown_connections 2022-11-29T18:04:05.150 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.138+0000 7f8830c04700 1 -- 172.21.15.35:0/3478575905 wait complete. 2022-11-29T18:04:05.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 Processor -- start 2022-11-29T18:04:05.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 -- start start 2022-11-29T18:04:05.157 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f882c0a0200 0x7f882c099c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.157 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0aea10 0x7f882c124d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.158 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f882c12dd90 0x7f882c12a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.158 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c1293d0 con 0x7f882c12dd90 2022-11-29T18:04:05.159 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c129800 con 0x7f882c0a0200 2022-11-29T18:04:05.159 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c129ae0 con 0x7f882c0aea10 2022-11-29T18:04:05.160 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0aea10 0x7f882c124d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.160 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0aea10 0x7f882c124d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57818/0 (socket says 172.21.15.35:57818) 2022-11-29T18:04:05.161 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 -- 172.21.15.35:0/832620519 learned_addr learned my addr 172.21.15.35:0/832620519 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:05.161 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882bfff700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f882c12dd90 0x7f882c12a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.161 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882b7fe700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f882c0a0200 0x7f882c099c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.162 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 -- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f882c0a0200 msgr2=0x7f882c099c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.162 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f882c0a0200 0x7f882c099c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.163 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 -- 172.21.15.35:0/832620519 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f882c12dd90 msgr2=0x7f882c12a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.163 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f882c12dd90 0x7f882c12a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.164 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 -- 172.21.15.35:0/832620519 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88140092a0 con 0x7f882c0aea10 2022-11-29T18:04:05.164 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882b7fe700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f882c0a0200 0x7f882c099c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-29T18:04:05.164 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f882affd700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0aea10 0x7f882c124d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f881c001ed0 tx=0x7f881c007c70).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:05.165 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f88237fe700 1 -- 172.21.15.35:0/832620519 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c0072e0 con 0x7f882c0aea10 2022-11-29T18:04:05.165 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f88237fe700 1 -- 172.21.15.35:0/832620519 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881c007440 con 0x7f882c0aea10 2022-11-29T18:04:05.166 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c12a690 con 0x7f882c0aea10 2022-11-29T18:04:05.166 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f88237fe700 1 -- 172.21.15.35:0/832620519 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c0057a0 con 0x7f882c0aea10 2022-11-29T18:04:05.167 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c09a700 con 0x7f882c0aea10 2022-11-29T18:04:05.167 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.142+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c12a860 con 0x7f882c0aea10 2022-11-29T18:04:05.168 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.146+0000 7f88237fe700 1 -- 172.21.15.35:0/832620519 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f881c010070 con 0x7f882c0aea10 2022-11-29T18:04:05.168 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.146+0000 7f88237fe700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:05.168 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.146+0000 7f88237fe700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:05.169 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.146+0000 7f88237fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:05.169 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.146+0000 7f88237fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:05.170 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.146+0000 7f88237fe700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8810084de0 0x7f88100872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.170 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.146+0000 7f88237fe700 1 -- 172.21.15.35:0/832620519 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3541+0+0 (secure 0 0 0) 0x7f881c0a6bc0 con 0x7f882c0aea10 2022-11-29T18:04:05.171 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.146+0000 7f882b7fe700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8810084de0 0x7f88100872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.171 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.150+0000 7f882b7fe700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8810084de0 0x7f88100872a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f881400b6c0 tx=0x7f8814017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:05.172 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.150+0000 7f88237fe700 1 -- 172.21.15.35:0/832620519 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f881c048980 con 0x7f882c0aea10 2022-11-29T18:04:05.308 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.306+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f882c046830 con 0x7f882c0aea10 2022-11-29T18:04:05.309 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.310+0000 7f88237fe700 1 -- 172.21.15.35:0/832620519 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v13) v1 ==== 84+0+143 (secure 0 0 0) 0x7f881c018030 con 0x7f882c0aea10 2022-11-29T18:04:05.310 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:04:05.310 INFO:teuthology.orchestra.run.smithi035.stdout:[ 2022-11-29T18:04:05.310 INFO:teuthology.orchestra.run.smithi035.stdout: { 2022-11-29T18:04:05.310 INFO:teuthology.orchestra.run.smithi035.stdout: "poolnum": 1, 2022-11-29T18:04:05.311 INFO:teuthology.orchestra.run.smithi035.stdout: "poolname": "rbd" 2022-11-29T18:04:05.311 INFO:teuthology.orchestra.run.smithi035.stdout: }, 2022-11-29T18:04:05.311 INFO:teuthology.orchestra.run.smithi035.stdout: { 2022-11-29T18:04:05.311 INFO:teuthology.orchestra.run.smithi035.stdout: "poolnum": 2, 2022-11-29T18:04:05.312 INFO:teuthology.orchestra.run.smithi035.stdout: "poolname": "device_health_metrics" 2022-11-29T18:04:05.312 INFO:teuthology.orchestra.run.smithi035.stdout: } 2022-11-29T18:04:05.312 INFO:teuthology.orchestra.run.smithi035.stdout:] 2022-11-29T18:04:05.313 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.310+0000 7f8830c04700 10 mgrc shutdown 2022-11-29T18:04:05.314 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.310+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8810084de0 msgr2=0x7f88100872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.314 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.310+0000 7f8830c04700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8810084de0 0x7f88100872a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f881400b6c0 tx=0x7f8814017000).stop 2022-11-29T18:04:05.315 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.310+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0aea10 msgr2=0x7f882c124d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.315 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.310+0000 7f8830c04700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0aea10 0x7f882c124d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f881c001ed0 tx=0x7f881c007c70).stop 2022-11-29T18:04:05.316 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.310+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 shutdown_connections 2022-11-29T18:04:05.316 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.314+0000 7f8830c04700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8810084de0 0x7f88100872a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.317 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.314+0000 7f8830c04700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f882c12dd90 0x7f882c12a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.317 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.314+0000 7f8830c04700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f882c0aea10 0x7f882c124d00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.317 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.314+0000 7f8830c04700 1 --2- 172.21.15.35:0/832620519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f882c0a0200 0x7f882c099c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.318 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.314+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 >> 172.21.15.35:0/832620519 conn(0x7f882c0619f0 msgr2=0x7f882c12e240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:05.318 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.314+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 shutdown_connections 2022-11-29T18:04:05.318 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.314+0000 7f8830c04700 1 -- 172.21.15.35:0/832620519 wait complete. 2022-11-29T18:04:05.328 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-29T18:04:05.328 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2022-11-29T18:04:05.436 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.434+0000 7f8ea759e700 1 Processor -- start 2022-11-29T18:04:05.436 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.434+0000 7f8ea759e700 1 -- start start 2022-11-29T18:04:05.440 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80ae630 0x7f8ea812e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.440 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f8ea80aea10 0x7f8ea813b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.440 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea813bbd0 con 0x7f8ea809ff40 2022-11-29T18:04:05.441 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea80a0ca0 con 0x7f8ea80aea10 2022-11-29T18:04:05.441 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea813be30 con 0x7f8ea80ae630 2022-11-29T18:04:05.442 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea5d9b700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f8ea80aea10 0x7f8ea813b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.442 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea659c700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80ae630 0x7f8ea812e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.442 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea5d9b700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f8ea80aea10 0x7f8ea813b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43830/0 (socket says 172.21.15.35:43830) 2022-11-29T18:04:05.443 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea5d9b700 1 -- 172.21.15.35:0/1694524186 learned_addr learned my addr 172.21.15.35:0/1694524186 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:05.443 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea5d9b700 1 -- 172.21.15.35:0/1694524186 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80ae630 msgr2=0x7f8ea812e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:05.443 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea5d9b700 1 --2- 172.21.15.35:0/1694524186 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80ae630 0x7f8ea812e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.443 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea5d9b700 1 -- 172.21.15.35:0/1694524186 >> v1:172.21.15.38:6789/0 conn(0x7f8ea809ff40 legacy=0x7f8ea80683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.444 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea5d9b700 1 -- 172.21.15.35:0/1694524186 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea813c0f0 con 0x7f8ea80aea10 2022-11-29T18:04:05.444 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea5d9b700 1 --2- 172.21.15.35:0/1694524186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea80aea10 0x7f8ea813b6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8e98006fd0 tx=0x7f8e980090f0).ready entity=mon.0 client_cookie=7af82ce18ab62410 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:05.444 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea559a700 1 -- 172.21.15.35:0/1694524186 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800b580 con 0x7f8ea80aea10 2022-11-29T18:04:05.444 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea559a700 1 -- 172.21.15.35:0/1694524186 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800b6e0 con 0x7f8ea80aea10 2022-11-29T18:04:05.445 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea559a700 1 -- 172.21.15.35:0/1694524186 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800ba60 con 0x7f8ea80aea10 2022-11-29T18:04:05.445 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 -- 172.21.15.35:0/1694524186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea80aea10 msgr2=0x7f8ea813b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.445 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 --2- 172.21.15.35:0/1694524186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea80aea10 0x7f8ea813b6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8e98006fd0 tx=0x7f8e980090f0).stop 2022-11-29T18:04:05.446 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 -- 172.21.15.35:0/1694524186 shutdown_connections 2022-11-29T18:04:05.446 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 --2- 172.21.15.35:0/1694524186 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea80aea10 0x7f8ea813b6e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.446 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 --2- 172.21.15.35:0/1694524186 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80ae630 0x7f8ea812e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.447 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 -- 172.21.15.35:0/1694524186 >> 172.21.15.35:0/1694524186 conn(0x7f8ea80619f0 msgr2=0x7f8ea8061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:05.447 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 -- 172.21.15.35:0/1694524186 shutdown_connections 2022-11-29T18:04:05.447 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.438+0000 7f8ea759e700 1 -- 172.21.15.35:0/1694524186 wait complete. 2022-11-29T18:04:05.450 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 Processor -- start 2022-11-29T18:04:05.451 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 -- start start 2022-11-29T18:04:05.451 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea809ff40 0x7f8ea80647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.451 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea80ae630 0x7f8ea812cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.452 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80a2550 0x7f8ea80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.452 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea812d040 con 0x7f8ea809ff40 2022-11-29T18:04:05.452 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea80a4130 con 0x7f8ea80ae630 2022-11-29T18:04:05.452 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8149c80 con 0x7f8ea80a2550 2022-11-29T18:04:05.452 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea809ff40 0x7f8ea80647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.453 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea809ff40 0x7f8ea80647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43836/0 (socket says 172.21.15.35:43836) 2022-11-29T18:04:05.453 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 -- 172.21.15.35:0/564767695 learned_addr learned my addr 172.21.15.35:0/564767695 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:05.453 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea6d9d700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80a2550 0x7f8ea80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.453 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea5d9b700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea80ae630 0x7f8ea812cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.454 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 -- 172.21.15.35:0/564767695 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80a2550 msgr2=0x7f8ea80a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.454 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80a2550 0x7f8ea80a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.454 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 -- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea80ae630 msgr2=0x7f8ea812cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:05.454 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea80ae630 0x7f8ea812cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:05.454 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 -- 172.21.15.35:0/564767695 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98009370 con 0x7f8ea809ff40 2022-11-29T18:04:05.455 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea659c700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea809ff40 0x7f8ea80647c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8e9000e960 tx=0x7f8e90009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:05.455 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8e9e7fc700 1 -- 172.21.15.35:0/564767695 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90012600 con 0x7f8ea809ff40 2022-11-29T18:04:05.455 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8e9e7fc700 1 -- 172.21.15.35:0/564767695 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e90012760 con 0x7f8ea809ff40 2022-11-29T18:04:05.456 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea812d4e0 con 0x7f8ea809ff40 2022-11-29T18:04:05.456 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8e9e7fc700 1 -- 172.21.15.35:0/564767695 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90012ae0 con 0x7f8ea809ff40 2022-11-29T18:04:05.456 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.442+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea80a3bc0 con 0x7f8ea809ff40 2022-11-29T18:04:05.456 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.446+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea8046830 con 0x7f8ea809ff40 2022-11-29T18:04:05.457 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.446+0000 7f8e9e7fc700 1 -- 172.21.15.35:0/564767695 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f8e90022070 con 0x7f8ea809ff40 2022-11-29T18:04:05.457 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.446+0000 7f8e9e7fc700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:05.457 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.450+0000 7f8e9e7fc700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:05.457 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.450+0000 7f8e9e7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:05.457 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.450+0000 7f8e9e7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:05.458 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.450+0000 7f8e9e7fc700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8e8c084de0 0x7f8e8c0872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:05.458 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.450+0000 7f8e9e7fc700 1 -- 172.21.15.35:0/564767695 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3541+0+0 (secure 0 0 0) 0x7f8e900a78b0 con 0x7f8ea809ff40 2022-11-29T18:04:05.458 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.450+0000 7f8ea5d9b700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8e8c084de0 0x7f8e8c0872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:05.458 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.450+0000 7f8ea5d9b700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8e8c084de0 0x7f8e8c0872a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8e98006fd0 tx=0x7f8e98017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:05.459 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.454+0000 7f8e9e7fc700 1 -- 172.21.15.35:0/564767695 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f8e9007bc90 con 0x7f8ea809ff40 2022-11-29T18:04:05.608 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.606+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f8ea809be30 con 0x7f8ea809ff40 2022-11-29T18:04:05.999 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:05.998+0000 7f8e9e7fc700 1 -- 172.21.15.35:0/564767695 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v14) v1 ==== 152+0+0 (secure 0 0 0) 0x7f8e9007f2b0 con 0x7f8ea809ff40 2022-11-29T18:04:06.005 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 10 mgrc shutdown 2022-11-29T18:04:06.005 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8e8c084de0 msgr2=0x7f8e8c0872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:06.006 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8e8c084de0 0x7f8e8c0872a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8e98006fd0 tx=0x7f8e98017000).stop 2022-11-29T18:04:06.006 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea809ff40 msgr2=0x7f8ea80647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:06.006 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea809ff40 0x7f8ea80647c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8e9000e960 tx=0x7f8e90009060).stop 2022-11-29T18:04:06.006 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 shutdown_connections 2022-11-29T18:04:06.006 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f8e8c084de0 0x7f8e8c0872a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.007 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f8ea80a2550 0x7f8ea80a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.007 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ea80ae630 0x7f8ea812cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.007 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 --2- 172.21.15.35:0/564767695 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f8ea809ff40 0x7f8ea80647c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.007 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 >> 172.21.15.35:0/564767695 conn(0x7f8ea80619f0 msgr2=0x7f8ea813aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:06.008 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 shutdown_connections 2022-11-29T18:04:06.008 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.002+0000 7f8ea759e700 1 -- 172.21.15.35:0/564767695 wait complete. 2022-11-29T18:04:06.008 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'cephfs_metadata' created 2022-11-29T18:04:06.019 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2022-11-29T18:04:06.134 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f976d6b8700 1 Processor -- start 2022-11-29T18:04:06.135 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f976d6b8700 1 -- start start 2022-11-29T18:04:06.138 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f976d6b8700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f97680acc30 0x7f9768099930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:06.138 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f976d6b8700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f97680ad010 0x7f97680a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:06.138 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f976d6b8700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976809e1a0 con 0x7f9768069c90 2022-11-29T18:04:06.139 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f976d6b8700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976809e6e0 con 0x7f97680ad010 2022-11-29T18:04:06.139 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f976d6b8700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976813e440 con 0x7f97680acc30 2022-11-29T18:04:06.139 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f975f7fe700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f97680ad010 0x7f97680a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:06.139 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f975f7fe700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f97680ad010 0x7f97680a8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43838/0 (socket says 172.21.15.35:43838) 2022-11-29T18:04:06.140 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f975f7fe700 1 -- 172.21.15.35:0/3431901643 learned_addr learned my addr 172.21.15.35:0/3431901643 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:06.140 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f9767fff700 1 --2- 172.21.15.35:0/3431901643 >> v2:172.21.15.38:3300/0 conn(0x7f97680acc30 0x7f9768099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:06.140 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f975f7fe700 1 -- 172.21.15.35:0/3431901643 >> v2:172.21.15.38:3300/0 conn(0x7f97680acc30 msgr2=0x7f9768099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:06.140 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f975f7fe700 1 --2- 172.21.15.35:0/3431901643 >> v2:172.21.15.38:3300/0 conn(0x7f97680acc30 0x7f9768099930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.141 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f975f7fe700 1 -- 172.21.15.35:0/3431901643 >> v1:172.21.15.38:6789/0 conn(0x7f9768069c90 legacy=0x7f976813dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:06.141 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f975f7fe700 1 -- 172.21.15.35:0/3431901643 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976813e580 con 0x7f97680ad010 2022-11-29T18:04:06.141 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f975f7fe700 1 --2- 172.21.15.35:0/3431901643 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680ad010 0x7f97680a8ac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f9768062990 tx=0x7f97500090f0).ready entity=mon.0 client_cookie=9e2b7dbb79ebb87c server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:06.141 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f97677fe700 1 -- 172.21.15.35:0/3431901643 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975000b520 con 0x7f97680ad010 2022-11-29T18:04:06.142 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f97677fe700 1 -- 172.21.15.35:0/3431901643 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975000b680 con 0x7f97680ad010 2022-11-29T18:04:06.142 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.134+0000 7f97677fe700 1 -- 172.21.15.35:0/3431901643 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975000ba00 con 0x7f97680ad010 2022-11-29T18:04:06.142 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- 172.21.15.35:0/3431901643 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680ad010 msgr2=0x7f97680a8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:06.142 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 --2- 172.21.15.35:0/3431901643 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680ad010 0x7f97680a8ac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f9768062990 tx=0x7f97500090f0).stop 2022-11-29T18:04:06.143 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- 172.21.15.35:0/3431901643 shutdown_connections 2022-11-29T18:04:06.143 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 --2- 172.21.15.35:0/3431901643 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680ad010 0x7f97680a8ac0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.143 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 --2- 172.21.15.35:0/3431901643 >> v2:172.21.15.38:3300/0 conn(0x7f97680acc30 0x7f9768099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.143 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- 172.21.15.35:0/3431901643 >> 172.21.15.35:0/3431901643 conn(0x7f9768062490 msgr2=0x7f97680ac520 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:06.144 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- 172.21.15.35:0/3431901643 shutdown_connections 2022-11-29T18:04:06.144 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- 172.21.15.35:0/3431901643 wait complete. 2022-11-29T18:04:06.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 Processor -- start 2022-11-29T18:04:06.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- start start 2022-11-29T18:04:06.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9768069c90 0x7f9768151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:06.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f97680acc30 0x7f9768092950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:06.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680aad40 0x7f9768091180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:06.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768152990 con 0x7f97680aad40 2022-11-29T18:04:06.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9768092500 con 0x7f9768069c90 2022-11-29T18:04:06.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976d6b8700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9768092e40 con 0x7f97680acc30 2022-11-29T18:04:06.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f97680acc30 0x7f9768092950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:06.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976ceb7700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680aad40 0x7f9768091180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:06.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f9767fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9768069c90 0x7f9768151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:06.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f97680acc30 0x7f9768092950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57826/0 (socket says 172.21.15.35:57826) 2022-11-29T18:04:06.150 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f9767fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9768069c90 0x7f9768151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.35:59082/0 (socket says 172.21.15.35:59082) 2022-11-29T18:04:06.150 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 -- 172.21.15.35:0/1456547132 learned_addr learned my addr 172.21.15.35:0/1456547132 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:06.150 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 -- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9768069c90 msgr2=0x7f9768151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:06.151 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9768069c90 0x7f9768151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.151 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 -- 172.21.15.35:0/1456547132 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680aad40 msgr2=0x7f9768091180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:06.151 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680aad40 0x7f9768091180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:06.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 -- 172.21.15.35:0/1456547132 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750009370 con 0x7f97680acc30 2022-11-29T18:04:06.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f976ceb7700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680aad40 0x7f9768091180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-29T18:04:06.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f975f7fe700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f97680acc30 0x7f9768092950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9750009130 tx=0x7f9750017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:06.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f9764ff9700 1 -- 172.21.15.35:0/1456547132 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975000be90 con 0x7f97680acc30 2022-11-29T18:04:06.153 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f9764ff9700 1 -- 172.21.15.35:0/1456547132 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975000b7f0 con 0x7f97680acc30 2022-11-29T18:04:06.153 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.138+0000 7f9764ff9700 1 -- 172.21.15.35:0/1456547132 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975001c650 con 0x7f97680acc30 2022-11-29T18:04:06.153 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9768152010 con 0x7f97680acc30 2022-11-29T18:04:06.153 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976814e250 con 0x7f97680acc30 2022-11-29T18:04:06.154 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97680ac070 con 0x7f97680acc30 2022-11-29T18:04:06.154 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f9764ff9700 1 -- 172.21.15.35:0/1456547132 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f9750022020 con 0x7f97680acc30 2022-11-29T18:04:06.154 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f9764ff9700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:06.154 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f9764ff9700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:06.155 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f9764ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:06.155 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f9764ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:06.155 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.142+0000 7f9764ff9700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f974c084e10 0x7f974c0872d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:06.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.146+0000 7f9764ff9700 1 -- 172.21.15.35:0/1456547132 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3911+0+0 (secure 0 0 0) 0x7f97500a8f40 con 0x7f97680acc30 2022-11-29T18:04:06.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.146+0000 7f9767fff700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f974c084e10 0x7f974c0872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:06.157 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.146+0000 7f9767fff700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f974c084e10 0x7f974c0872d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f9758009c30 tx=0x7f9758009060).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:06.158 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.150+0000 7f9764ff9700 1 -- 172.21.15.35:0/1456547132 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f97680ac070 con 0x7f97680acc30 2022-11-29T18:04:06.312 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:06.310+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f97680ac070 con 0x7f97680acc30 2022-11-29T18:04:07.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.010+0000 7f9764ff9700 1 -- 172.21.15.35:0/1456547132 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v15) v1 ==== 170+0+0 (secure 0 0 0) 0x7f97680ac070 con 0x7f97680acc30 2022-11-29T18:04:07.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.014+0000 7f976d6b8700 10 mgrc shutdown 2022-11-29T18:04:07.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.014+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f974c084e10 msgr2=0x7f974c0872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.022 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.014+0000 7f976d6b8700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f974c084e10 0x7f974c0872d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f9758009c30 tx=0x7f9758009060).stop 2022-11-29T18:04:07.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.014+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f97680acc30 msgr2=0x7f9768092950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.014+0000 7f976d6b8700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f97680acc30 0x7f9768092950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9750009130 tx=0x7f9750017000).stop 2022-11-29T18:04:07.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.018+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 shutdown_connections 2022-11-29T18:04:07.023 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.018+0000 7f976d6b8700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f974c084e10 0x7f974c0872d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.018+0000 7f976d6b8700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f97680aad40 0x7f9768091180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.018+0000 7f976d6b8700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f97680acc30 0x7f9768092950 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.018+0000 7f976d6b8700 1 --2- 172.21.15.35:0/1456547132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9768069c90 0x7f9768151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.024 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.018+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 >> 172.21.15.35:0/1456547132 conn(0x7f9768062490 msgr2=0x7f9768062c80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:07.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.018+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 shutdown_connections 2022-11-29T18:04:07.025 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.018+0000 7f976d6b8700 1 -- 172.21.15.35:0/1456547132 wait complete. 2022-11-29T18:04:07.025 INFO:teuthology.orchestra.run.smithi035.stderr:pool 'cephfs_data' created 2022-11-29T18:04:07.032 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-11-29T18:04:07.143 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9be09db700 1 Processor -- start 2022-11-29T18:04:07.143 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9be09db700 1 -- start start 2022-11-29T18:04:07.146 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9be09db700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f9bdc134760 0x7f9bdc068ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.146 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9be09db700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9bdc13f8f0 0x7f9bdc141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.146 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9be09db700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bdc0685b0 con 0x7f9bdc0a01d0 2022-11-29T18:04:07.146 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9be09db700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bdc134b40 con 0x7f9bdc134760 2022-11-29T18:04:07.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9be09db700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bdc067d40 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9bda59c700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f9bdc0a01d0 0x7f9bdc0647f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50342/0 (socket says 172.21.15.35:50342) 2022-11-29T18:04:07.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9bda59c700 1 -- 172.21.15.35:0/1854687598 learned_addr learned my addr 172.21.15.35:0/1854687598 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:07.147 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.142+0000 7f9bdad9d700 1 --2- 172.21.15.35:0/1854687598 >> v2:172.21.15.35:3300/0 conn(0x7f9bdc134760 0x7f9bdc068ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bdb59e700 1 --2- 172.21.15.35:0/1854687598 >> v2:172.21.15.38:3300/0 conn(0x7f9bdc13f8f0 0x7f9bdc141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bdad9d700 1 -- 172.21.15.35:0/1854687598 >> v2:172.21.15.38:3300/0 conn(0x7f9bdc13f8f0 msgr2=0x7f9bdc141ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:07.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bdad9d700 1 --2- 172.21.15.35:0/1854687598 >> v2:172.21.15.38:3300/0 conn(0x7f9bdc13f8f0 0x7f9bdc141ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.148 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bdad9d700 1 -- 172.21.15.35:0/1854687598 >> v1:172.21.15.35:6789/0 conn(0x7f9bdc0a01d0 legacy=0x7f9bdc0647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bd9d9b700 1 -- 172.21.15.35:0/1854687598 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905078856 0 0) 0x7f9bdc0685b0 con 0x7f9bdc0a01d0 2022-11-29T18:04:07.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bdad9d700 1 -- 172.21.15.35:0/1854687598 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc09fd10 con 0x7f9bdc134760 2022-11-29T18:04:07.149 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bdad9d700 1 --2- 172.21.15.35:0/1854687598 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc134760 0x7f9bdc068ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9bdc069be0 tx=0x7f9bcc009060).ready entity=mon.0 client_cookie=4e629d2f9ef2153b server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.150 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bd9d9b700 1 -- 172.21.15.35:0/1854687598 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00f510 con 0x7f9bdc134760 2022-11-29T18:04:07.151 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bd9d9b700 1 -- 172.21.15.35:0/1854687598 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc00f670 con 0x7f9bdc134760 2022-11-29T18:04:07.151 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9bd9d9b700 1 -- 172.21.15.35:0/1854687598 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00f9f0 con 0x7f9bdc134760 2022-11-29T18:04:07.151 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9be09db700 1 -- 172.21.15.35:0/1854687598 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc134760 msgr2=0x7f9bdc068ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.151 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9be09db700 1 --2- 172.21.15.35:0/1854687598 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc134760 0x7f9bdc068ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9bdc069be0 tx=0x7f9bcc009060).stop 2022-11-29T18:04:07.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9be09db700 1 -- 172.21.15.35:0/1854687598 shutdown_connections 2022-11-29T18:04:07.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9be09db700 1 --2- 172.21.15.35:0/1854687598 >> v2:172.21.15.38:3300/0 conn(0x7f9bdc13f8f0 0x7f9bdc141ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9be09db700 1 --2- 172.21.15.35:0/1854687598 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc134760 0x7f9bdc068ab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.152 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9be09db700 1 -- 172.21.15.35:0/1854687598 >> 172.21.15.35:0/1854687598 conn(0x7f9bdc0ae040 msgr2=0x7f9bdc0640e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:07.153 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9be09db700 1 -- 172.21.15.35:0/1854687598 shutdown_connections 2022-11-29T18:04:07.153 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.146+0000 7f9be09db700 1 -- 172.21.15.35:0/1854687598 wait complete. 2022-11-29T18:04:07.153 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9be09db700 1 Processor -- start 2022-11-29T18:04:07.153 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9be09db700 1 -- start start 2022-11-29T18:04:07.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9be09db700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f9bdc0a01d0 0x7f9bdc09c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9be09db700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc134760 0x7f9bdc131710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9be09db700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc13f8f0 0x7f9bdc0970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.156 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9be09db700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc09c500 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.157 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9be09db700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc13c850 con 0x7f9bdc134760 2022-11-29T18:04:07.157 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9be09db700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc131c00 con 0x7f9bdc0a01d0 2022-11-29T18:04:07.157 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdad9d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f9bdc0a01d0 0x7f9bdc09c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.157 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdad9d700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f9bdc0a01d0 0x7f9bdc09c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57832/0 (socket says 172.21.15.35:57832) 2022-11-29T18:04:07.158 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdb59e700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc13f8f0 0x7f9bdc0970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.158 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdad9d700 1 -- 172.21.15.35:0/3321537924 learned_addr learned my addr 172.21.15.35:0/3321537924 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:07.158 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bda59c700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc134760 0x7f9bdc131710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.158 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdb59e700 1 -- 172.21.15.35:0/3321537924 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f9bdc0a01d0 msgr2=0x7f9bdc09c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.158 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdb59e700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f9bdc0a01d0 0x7f9bdc09c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.159 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdb59e700 1 -- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc134760 msgr2=0x7f9bdc131710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.159 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdb59e700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc134760 0x7f9bdc131710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.159 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdb59e700 1 -- 172.21.15.35:0/3321537924 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc0092a0 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.159 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.150+0000 7f9bdad9d700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f9bdc0a01d0 0x7f9bdc09c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-29T18:04:07.160 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9bdb59e700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc13f8f0 0x7f9bdc0970c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f9bd400ecf0 tx=0x7f9bd400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.160 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9bc2ffd700 1 -- 172.21.15.35:0/3321537924 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400c980 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.160 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9bc2ffd700 1 -- 172.21.15.35:0/3321537924 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd400f040 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.160 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9bc2ffd700 1 -- 172.21.15.35:0/3321537924 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400ccf0 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.161 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc097a00 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.161 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bdc13cce0 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.161 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bdc097cc0 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.162 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9bc2ffd700 1 -- 172.21.15.35:0/3321537924 <== mon.0 v2:172.21.15.35:3300/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f9bd4013610 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.162 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.154+0000 7f9bc2ffd700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:07.163 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.158+0000 7f9bc2ffd700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:07.163 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.158+0000 7f9bc2ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:07.163 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.158+0000 7f9bc2ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:07.163 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.158+0000 7f9bc2ffd700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9bbc084de0 0x7f9bbc0872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.164 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.158+0000 7f9bc2ffd700 1 -- 172.21.15.35:0/3321537924 <== mon.0 v2:172.21.15.35:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4293+0+0 (secure 0 0 0) 0x7f9bd40a7f60 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.164 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.158+0000 7f9bdad9d700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9bbc084de0 0x7f9bbc0872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.164 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.158+0000 7f9bdad9d700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9bbc084de0 0x7f9bbc0872a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f9bcc006be0 tx=0x7f9bcc017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.165 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.162+0000 7f9bc2ffd700 1 -- 172.21.15.35:0/3321537924 <== mon.0 v2:172.21.15.35:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f9bd407c010 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.351 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.350+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9bdc13c990 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.366 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.366+0000 7f9bc2ffd700 1 -- 172.21.15.35:0/3321537924 <== mon.0 v2:172.21.15.35:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9bd407f630 con 0x7f9bdc13f8f0 2022-11-29T18:04:07.372 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 10 mgrc shutdown 2022-11-29T18:04:07.373 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9bbc084de0 msgr2=0x7f9bbc0872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.373 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9bbc084de0 0x7f9bbc0872a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f9bcc006be0 tx=0x7f9bcc017000).stop 2022-11-29T18:04:07.374 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc13f8f0 msgr2=0x7f9bdc0970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.374 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc13f8f0 0x7f9bdc0970c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f9bd400ecf0 tx=0x7f9bd400b040).stop 2022-11-29T18:04:07.375 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 shutdown_connections 2022-11-29T18:04:07.375 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9bbc084de0 0x7f9bbc0872a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.375 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f9bdc13f8f0 0x7f9bdc0970c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.376 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bdc134760 0x7f9bdc131710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.376 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 --2- 172.21.15.35:0/3321537924 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f9bdc0a01d0 0x7f9bdc09c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.377 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 >> 172.21.15.35:0/3321537924 conn(0x7f9bdc0ae040 msgr2=0x7f9bdc133660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:07.377 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 shutdown_connections 2022-11-29T18:04:07.378 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.370+0000 7f9be09db700 1 -- 172.21.15.35:0/3321537924 wait complete. 2022-11-29T18:04:07.378 INFO:teuthology.orchestra.run.smithi035.stderr:new fs with metadata pool 3 and data pool 4 2022-11-29T18:04:07.384 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-29T18:04:07.485 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.482+0000 7f689a054700 1 Processor -- start 2022-11-29T18:04:07.485 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f689a054700 1 -- start start 2022-11-29T18:04:07.488 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f689a054700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f689405d820 0x7f6894138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.488 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f689a054700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f6894099fd0 0x7f689413b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.488 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f689a054700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689409f070 con 0x7f689409a3b0 2022-11-29T18:04:07.489 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f689a054700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689409d530 con 0x7f6894099fd0 2022-11-29T18:04:07.489 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f689a054700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689409d7d0 con 0x7f689405d820 2022-11-29T18:04:07.489 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6898851700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f6894099fd0 0x7f689413b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.489 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6898851700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f6894099fd0 0x7f689413b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43880/0 (socket says 172.21.15.35:43880) 2022-11-29T18:04:07.490 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6899853700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f689409a3b0 0x7f689413bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50354/0 (socket says 172.21.15.35:50354) 2022-11-29T18:04:07.490 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6898851700 1 -- 172.21.15.35:0/2740032514 learned_addr learned my addr 172.21.15.35:0/2740032514 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:07.490 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6899052700 1 --2- 172.21.15.35:0/2740032514 >> v2:172.21.15.38:3300/0 conn(0x7f689405d820 0x7f6894138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.490 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6898851700 1 -- 172.21.15.35:0/2740032514 >> v2:172.21.15.38:3300/0 conn(0x7f689405d820 msgr2=0x7f6894138cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:07.490 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6898851700 1 --2- 172.21.15.35:0/2740032514 >> v2:172.21.15.38:3300/0 conn(0x7f689405d820 0x7f6894138cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.491 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6898851700 1 -- 172.21.15.35:0/2740032514 >> v1:172.21.15.35:6789/0 conn(0x7f689409a3b0 legacy=0x7f689413bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.491 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6898851700 1 -- 172.21.15.35:0/2740032514 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689405dc00 con 0x7f6894099fd0 2022-11-29T18:04:07.491 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6883fff700 1 -- 172.21.15.35:0/2740032514 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834403184 0 0) 0x7f689409f070 con 0x7f689409a3b0 2022-11-29T18:04:07.491 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6898851700 1 --2- 172.21.15.35:0/2740032514 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6894099fd0 0x7f689413b780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f689409ff70 tx=0x7f68840090f0).ready entity=mon.0 client_cookie=c1c69597a1f71b8c server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.492 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6883fff700 1 -- 172.21.15.35:0/2740032514 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400b5b0 con 0x7f6894099fd0 2022-11-29T18:04:07.492 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6883fff700 1 -- 172.21.15.35:0/2740032514 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688400b710 con 0x7f6894099fd0 2022-11-29T18:04:07.492 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.486+0000 7f6883fff700 1 -- 172.21.15.35:0/2740032514 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688400ba90 con 0x7f6894099fd0 2022-11-29T18:04:07.495 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- 172.21.15.35:0/2740032514 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6894099fd0 msgr2=0x7f689413b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.496 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 --2- 172.21.15.35:0/2740032514 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6894099fd0 0x7f689413b780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f689409ff70 tx=0x7f68840090f0).stop 2022-11-29T18:04:07.496 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- 172.21.15.35:0/2740032514 shutdown_connections 2022-11-29T18:04:07.496 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 --2- 172.21.15.35:0/2740032514 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f6894099fd0 0x7f689413b780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.496 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 --2- 172.21.15.35:0/2740032514 >> v2:172.21.15.38:3300/0 conn(0x7f689405d820 0x7f6894138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.497 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- 172.21.15.35:0/2740032514 >> 172.21.15.35:0/2740032514 conn(0x7f68940624c0 msgr2=0x7f68940acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:07.497 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- 172.21.15.35:0/2740032514 shutdown_connections 2022-11-29T18:04:07.497 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- 172.21.15.35:0/2740032514 wait complete. 2022-11-29T18:04:07.497 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 Processor -- start 2022-11-29T18:04:07.497 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- start start 2022-11-29T18:04:07.498 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f689405d820 0x7f68940686f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.498 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6894099fd0 0x7f68940a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.498 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f689409a3b0 0x7f689414d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.498 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894068be0 con 0x7f689409a3b0 2022-11-29T18:04:07.498 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68940aa600 con 0x7f6894099fd0 2022-11-29T18:04:07.499 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f689a054700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68940975f0 con 0x7f689405d820 2022-11-29T18:04:07.499 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f6899052700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f689405d820 0x7f68940686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.499 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f6899052700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f689405d820 0x7f68940686f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57848/0 (socket says 172.21.15.35:57848) 2022-11-29T18:04:07.499 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f6899853700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f689409a3b0 0x7f689414d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.490+0000 7f6899052700 1 -- 172.21.15.35:0/2168493925 learned_addr learned my addr 172.21.15.35:0/2168493925 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:07.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f6898851700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6894099fd0 0x7f68940a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f6899052700 1 -- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6894099fd0 msgr2=0x7f68940a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f6899052700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6894099fd0 0x7f68940a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f6899052700 1 -- 172.21.15.35:0/2168493925 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f689409a3b0 msgr2=0x7f689414d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.501 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f6899052700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f689409a3b0 0x7f689414d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.501 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f6899052700 1 -- 172.21.15.35:0/2168493925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6884009370 con 0x7f689405d820 2022-11-29T18:04:07.501 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f6899052700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f689405d820 0x7f68940686f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f689000e960 tx=0x7f6890009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.501 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f68817fa700 1 -- 172.21.15.35:0/2168493925 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890009510 con 0x7f689405d820 2022-11-29T18:04:07.502 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f68817fa700 1 -- 172.21.15.35:0/2168493925 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6890012600 con 0x7f689405d820 2022-11-29T18:04:07.502 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68940978b0 con 0x7f689405d820 2022-11-29T18:04:07.502 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f68817fa700 1 -- 172.21.15.35:0/2168493925 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890012980 con 0x7f689405d820 2022-11-29T18:04:07.502 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68940a0ec0 con 0x7f689405d820 2022-11-29T18:04:07.504 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.494+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68940980e0 con 0x7f689405d820 2022-11-29T18:04:07.504 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f68817fa700 1 -- 172.21.15.35:0/2168493925 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f6890022070 con 0x7f689405d820 2022-11-29T18:04:07.504 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f68817fa700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:07.504 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f68817fa700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:07.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f68817fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:07.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f68817fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:07.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f68817fa700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6874084de0 0x7f68740872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f68817fa700 1 -- 172.21.15.35:0/2168493925 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f68900a79a0 con 0x7f689405d820 2022-11-29T18:04:07.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f6898851700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6874084de0 0x7f68740872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.506 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.498+0000 7f6898851700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6874084de0 0x7f68740872a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f6884009130 tx=0x7f6884017000).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.506 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.502+0000 7f68817fa700 1 -- 172.21.15.35:0/2168493925 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f689007bb00 con 0x7f689405d820 2022-11-29T18:04:07.656 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.654+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6894046830 con 0x7f689405d820 2022-11-29T18:04:07.658 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f68817fa700 1 -- 172.21.15.35:0/2168493925 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12233 (secure 0 0 0) 0x7f689007f120 con 0x7f689405d820 2022-11-29T18:04:07.658 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:04:07.659 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":16,"fsid":"338b2b5b-2e59-4e89-9321-f2b909af04fd","created":"2022-11-29T18:03:52.771747+0000","modified":"2022-11-29T18:04:07.354481+0000","last_up_change":"2022-11-29T18:04:00.802110+0000","last_in_change":"2022-11-29T18:03:53.874582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"rbd","create_time":"2022-11-29T18:04:01.750436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"device_health_metrics","create_time":"2022-11-29T18:04:01.898453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-11-29T18:04:05.674457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-11-29T18:04:06.378188+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"228fad84-4a8f-44ce-ba6d-f0433a5a22ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":22147},{"type":"v1","addr":"172.21.15.35:6801","nonce":22147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":22147},{"type":"v1","addr":"172.21.15.35:6803","nonce":22147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":22147},{"type":"v1","addr":"172.21.15.35:6807","nonce":22147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":22147},{"type":"v1","addr":"172.21.15.35:6805","nonce":22147}]},"public_addr":"172.21.15.35:6801/22147","cluster_addr":"172.21.15.35:6803/22147","heartbeat_back_addr":"172.21.15.35:6807/22147","heartbeat_front_addr":"172.21.15.35:6805/22147","state":["exists","up"]},{"osd":1,"uuid":"d58467d0-703a-4da6-980d-30bf1051838b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24521},{"type":"v1","addr":"172.21.15.38:6803","nonce":24521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6805","nonce":24521},{"type":"v1","addr":"172.21.15.38:6806","nonce":24521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6811","nonce":24521},{"type":"v1","addr":"172.21.15.38:6815","nonce":24521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24521},{"type":"v1","addr":"172.21.15.38:6809","nonce":24521}]},"public_addr":"172.21.15.38:6803/24521","cluster_addr":"172.21.15.38:6806/24521","heartbeat_back_addr":"172.21.15.38:6815/24521","heartbeat_front_addr":"172.21.15.38:6809/24521","state":["exists","up"]},{"osd":2,"uuid":"34a91d7b-b17b-4400-b2f3-0de86be92ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6813","nonce":24523},{"type":"v1","addr":"172.21.15.38:6816","nonce":24523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24523},{"type":"v1","addr":"172.21.15.38:6819","nonce":24523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24523},{"type":"v1","addr":"172.21.15.38:6823","nonce":24523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24523},{"type":"v1","addr":"172.21.15.38:6821","nonce":24523}]},"public_addr":"172.21.15.38:6816/24523","cluster_addr":"172.21.15.38:6819/24523","heartbeat_back_addr":"172.21.15.38:6823/24523","heartbeat_front_addr":"172.21.15.38:6821/24523","state":["exists","up"]},{"osd":3,"uuid":"c7d3a292-4f7e-456d-ad04-0a67f33eb539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24522},{"type":"v1","addr":"172.21.15.38:6801","nonce":24522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24522},{"type":"v1","addr":"172.21.15.38:6807","nonce":24522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24522},{"type":"v1","addr":"172.21.15.38:6817","nonce":24522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24522},{"type":"v1","addr":"172.21.15.38:6812","nonce":24522}]},"public_addr":"172.21.15.38:6801/24522","cluster_addr":"172.21.15.38:6807/24522","heartbeat_back_addr":"172.21.15.38:6817/24522","heartbeat_front_addr":"172.21.15.38:6812/24522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.171690+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.165049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.245376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.051689+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-29T18:04:07.660 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 10 mgrc shutdown 2022-11-29T18:04:07.661 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6874084de0 msgr2=0x7f68740872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.661 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6874084de0 0x7f68740872a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f6884009130 tx=0x7f6884017000).stop 2022-11-29T18:04:07.661 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f689405d820 msgr2=0x7f68940686f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.661 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f689405d820 0x7f68940686f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f689000e960 tx=0x7f6890009060).stop 2022-11-29T18:04:07.662 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 shutdown_connections 2022-11-29T18:04:07.662 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f6874084de0 0x7f68740872a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.662 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f689409a3b0 0x7f689414d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.662 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6894099fd0 0x7f68940a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.662 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 --2- 172.21.15.35:0/2168493925 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f689405d820 0x7f68940686f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.663 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 >> 172.21.15.35:0/2168493925 conn(0x7f68940624c0 msgr2=0x7f68940abb00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:07.663 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 shutdown_connections 2022-11-29T18:04:07.663 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.658+0000 7f689a054700 1 -- 172.21.15.35:0/2168493925 wait complete. 2022-11-29T18:04:07.672 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-29T18:04:07.778 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac099f2700 1 Processor -- start 2022-11-29T18:04:07.779 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac099f2700 1 -- start start 2022-11-29T18:04:07.781 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac099f2700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fac04069b90 0x7fac04068c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.781 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac099f2700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04069140 0x7fac04141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.781 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac099f2700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac040696d0 con 0x7fac04137da0 2022-11-29T18:04:07.782 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac099f2700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac04138180 con 0x7fac04069b90 2022-11-29T18:04:07.782 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac099f2700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac041370f0 con 0x7fac04069140 2022-11-29T18:04:07.782 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac091f1700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04069140 0x7fac04141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.782 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac091f1700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04069140 0x7fac04141ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57852/0 (socket says 172.21.15.35:57852) 2022-11-29T18:04:07.783 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac091f1700 1 -- 172.21.15.35:0/803017724 learned_addr learned my addr 172.21.15.35:0/803017724 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:07.783 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac03fff700 1 --2- 172.21.15.35:0/803017724 >> v2:172.21.15.38:3300/0 conn(0x7fac04069b90 0x7fac04068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.783 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac091f1700 1 -- 172.21.15.35:0/803017724 >> v2:172.21.15.38:3300/0 conn(0x7fac04069b90 msgr2=0x7fac04068c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:07.783 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac091f1700 1 --2- 172.21.15.35:0/803017724 >> v2:172.21.15.38:3300/0 conn(0x7fac04069b90 0x7fac04068c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.784 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac091f1700 1 -- 172.21.15.35:0/803017724 >> v1:172.21.15.38:6789/0 conn(0x7fac04137da0 legacy=0x7fac040adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.784 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac091f1700 1 -- 172.21.15.35:0/803017724 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac04137350 con 0x7fac04069140 2022-11-29T18:04:07.784 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac091f1700 1 --2- 172.21.15.35:0/803017724 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04069140 0x7fac04141ac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fabfc009f20 tx=0x7fabfc009840).ready entity=mon.2 client_cookie=70113cf6372f3c4e server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.784 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac037fe700 1 -- 172.21.15.35:0/803017724 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00e070 con 0x7fac04069140 2022-11-29T18:04:07.785 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac037fe700 1 -- 172.21.15.35:0/803017724 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00ba60 con 0x7fac04069140 2022-11-29T18:04:07.785 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.778+0000 7fac037fe700 1 -- 172.21.15.35:0/803017724 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00bde0 con 0x7fac04069140 2022-11-29T18:04:07.789 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 -- 172.21.15.35:0/803017724 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04069140 msgr2=0x7fac04141ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.789 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 --2- 172.21.15.35:0/803017724 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04069140 0x7fac04141ac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fabfc009f20 tx=0x7fabfc009840).stop 2022-11-29T18:04:07.789 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 -- 172.21.15.35:0/803017724 shutdown_connections 2022-11-29T18:04:07.789 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 --2- 172.21.15.35:0/803017724 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04069140 0x7fac04141ac0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.790 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 --2- 172.21.15.35:0/803017724 >> v2:172.21.15.38:3300/0 conn(0x7fac04069b90 0x7fac04068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.790 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 -- 172.21.15.35:0/803017724 >> 172.21.15.35:0/803017724 conn(0x7fac040635d0 msgr2=0x7fac040ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:07.790 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 -- 172.21.15.35:0/803017724 shutdown_connections 2022-11-29T18:04:07.791 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 -- 172.21.15.35:0/803017724 wait complete. 2022-11-29T18:04:07.791 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 Processor -- start 2022-11-29T18:04:07.791 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.782+0000 7fac099f2700 1 -- start start 2022-11-29T18:04:07.791 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac04069140 0x7fac040a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.792 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fac04069b90 0x7fac041339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.792 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04137da0 0x7fac0409c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.792 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac040a5c30 con 0x7fac04069b90 2022-11-29T18:04:07.792 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0414ba80 con 0x7fac04069140 2022-11-29T18:04:07.792 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0414bd40 con 0x7fac04137da0 2022-11-29T18:04:07.793 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04137da0 0x7fac0409c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.793 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac03fff700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fac04069b90 0x7fac041339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.793 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04137da0 0x7fac0409c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57860/0 (socket says 172.21.15.35:57860) 2022-11-29T18:04:07.793 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 -- 172.21.15.35:0/678039933 learned_addr learned my addr 172.21.15.35:0/678039933 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:07.793 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac089f0700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac04069140 0x7fac040a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.794 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 -- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac04069140 msgr2=0x7fac040a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.794 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac04069140 0x7fac040a60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.794 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 -- 172.21.15.35:0/678039933 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fac04069b90 msgr2=0x7fac041339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.794 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fac04069b90 0x7fac041339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.795 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 -- 172.21.15.35:0/678039933 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc009ae0 con 0x7fac04137da0 2022-11-29T18:04:07.795 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac091f1700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04137da0 0x7fac0409c750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fabfc009880 tx=0x7fac040655c0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.795 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac00ff9700 1 -- 172.21.15.35:0/678039933 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0092e0 con 0x7fac04137da0 2022-11-29T18:04:07.795 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac00ff9700 1 -- 172.21.15.35:0/678039933 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc01e470 con 0x7fac04137da0 2022-11-29T18:04:07.795 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac040a65d0 con 0x7fac04137da0 2022-11-29T18:04:07.796 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac00ff9700 1 -- 172.21.15.35:0/678039933 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc01e830 con 0x7fac04137da0 2022-11-29T18:04:07.796 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac040ac200 con 0x7fac04137da0 2022-11-29T18:04:07.796 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.786+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac040a5e00 con 0x7fac04137da0 2022-11-29T18:04:07.797 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac00ff9700 1 -- 172.21.15.35:0/678039933 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7fabfc025030 con 0x7fac04137da0 2022-11-29T18:04:07.798 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac00ff9700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:07.798 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac00ff9700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:07.798 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac00ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:07.798 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac00ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:07.799 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac00ff9700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fabe4084de0 0x7fabe40872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:07.799 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac00ff9700 1 -- 172.21.15.35:0/678039933 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fabfc0a8e10 con 0x7fac04137da0 2022-11-29T18:04:07.799 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac089f0700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fabe4084de0 0x7fabe40872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:07.799 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.790+0000 7fac089f0700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fabe4084de0 0x7fabe40872a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fabf8003860 tx=0x7fabf8007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:07.800 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.798+0000 7fac00ff9700 1 -- 172.21.15.35:0/678039933 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7fac040a5e00 con 0x7fac04137da0 2022-11-29T18:04:07.955 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.954+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac04046830 con 0x7fac04137da0 2022-11-29T18:04:07.956 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.954+0000 7fac00ff9700 1 -- 172.21.15.35:0/678039933 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12233 (secure 0 0 0) 0x7fabfc023070 con 0x7fac04137da0 2022-11-29T18:04:07.957 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:04:07.957 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":16,"fsid":"338b2b5b-2e59-4e89-9321-f2b909af04fd","created":"2022-11-29T18:03:52.771747+0000","modified":"2022-11-29T18:04:07.354481+0000","last_up_change":"2022-11-29T18:04:00.802110+0000","last_in_change":"2022-11-29T18:03:53.874582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"rbd","create_time":"2022-11-29T18:04:01.750436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":"device_health_metrics","create_time":"2022-11-29T18:04:01.898453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2022-11-29T18:04:05.674457+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2022-11-29T18:04:06.378188+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"228fad84-4a8f-44ce-ba6d-f0433a5a22ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6800","nonce":22147},{"type":"v1","addr":"172.21.15.35:6801","nonce":22147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6802","nonce":22147},{"type":"v1","addr":"172.21.15.35:6803","nonce":22147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6806","nonce":22147},{"type":"v1","addr":"172.21.15.35:6807","nonce":22147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6804","nonce":22147},{"type":"v1","addr":"172.21.15.35:6805","nonce":22147}]},"public_addr":"172.21.15.35:6801/22147","cluster_addr":"172.21.15.35:6803/22147","heartbeat_back_addr":"172.21.15.35:6807/22147","heartbeat_front_addr":"172.21.15.35:6805/22147","state":["exists","up"]},{"osd":1,"uuid":"d58467d0-703a-4da6-980d-30bf1051838b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":24521},{"type":"v1","addr":"172.21.15.38:6803","nonce":24521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6805","nonce":24521},{"type":"v1","addr":"172.21.15.38:6806","nonce":24521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6811","nonce":24521},{"type":"v1","addr":"172.21.15.38:6815","nonce":24521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":24521},{"type":"v1","addr":"172.21.15.38:6809","nonce":24521}]},"public_addr":"172.21.15.38:6803/24521","cluster_addr":"172.21.15.38:6806/24521","heartbeat_back_addr":"172.21.15.38:6815/24521","heartbeat_front_addr":"172.21.15.38:6809/24521","state":["exists","up"]},{"osd":2,"uuid":"34a91d7b-b17b-4400-b2f3-0de86be92ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6813","nonce":24523},{"type":"v1","addr":"172.21.15.38:6816","nonce":24523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":24523},{"type":"v1","addr":"172.21.15.38:6819","nonce":24523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":24523},{"type":"v1","addr":"172.21.15.38:6823","nonce":24523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":24523},{"type":"v1","addr":"172.21.15.38:6821","nonce":24523}]},"public_addr":"172.21.15.38:6816/24523","cluster_addr":"172.21.15.38:6819/24523","heartbeat_back_addr":"172.21.15.38:6823/24523","heartbeat_front_addr":"172.21.15.38:6821/24523","state":["exists","up"]},{"osd":3,"uuid":"c7d3a292-4f7e-456d-ad04-0a67f33eb539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":24522},{"type":"v1","addr":"172.21.15.38:6801","nonce":24522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":24522},{"type":"v1","addr":"172.21.15.38:6807","nonce":24522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":24522},{"type":"v1","addr":"172.21.15.38:6817","nonce":24522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":24522},{"type":"v1","addr":"172.21.15.38:6812","nonce":24522}]},"public_addr":"172.21.15.38:6801/24522","cluster_addr":"172.21.15.38:6807/24522","heartbeat_back_addr":"172.21.15.38:6817/24522","heartbeat_front_addr":"172.21.15.38:6812/24522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.171690+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.165049+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.245376+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2022-11-29T18:03:59.051689+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-29T18:04:07.959 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 10 mgrc shutdown 2022-11-29T18:04:07.960 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fabe4084de0 msgr2=0x7fabe40872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.960 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fabe4084de0 0x7fabe40872a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fabf8003860 tx=0x7fabf8007d40).stop 2022-11-29T18:04:07.960 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04137da0 msgr2=0x7fac0409c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:07.961 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04137da0 0x7fac0409c750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fabfc009880 tx=0x7fac040655c0).stop 2022-11-29T18:04:07.961 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 shutdown_connections 2022-11-29T18:04:07.961 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fabe4084de0 0x7fabe40872a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.961 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fac04137da0 0x7fac0409c750 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.962 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fac04069b90 0x7fac041339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.962 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 --2- 172.21.15.35:0/678039933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac04069140 0x7fac040a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:07.962 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 >> 172.21.15.35:0/678039933 conn(0x7fac040635d0 msgr2=0x7fac04064b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:07.962 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 shutdown_connections 2022-11-29T18:04:07.963 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:07.958+0000 7fac099f2700 1 -- 172.21.15.35:0/678039933 wait complete. 2022-11-29T18:04:07.972 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2022-11-29T18:04:08.087 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 Processor -- start 2022-11-29T18:04:08.087 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- start start 2022-11-29T18:04:08.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f927c0aced0 0x7f927c13de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f927c099b00 0x7f927c13e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c09cc70 con 0x7f927c142200 2022-11-29T18:04:08.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- --> v2:172.21.15.35:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c0ad2b0 con 0x7f927c099b00 2022-11-29T18:04:08.088 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c064560 con 0x7f927c0aced0 2022-11-29T18:04:08.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9279d9b700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f927c099b00 0x7f927c13e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9279d9b700 1 --2- >> v2:172.21.15.35:3300/0 conn(0x7f927c099b00 0x7f927c13e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43926/0 (socket says 172.21.15.35:43926) 2022-11-29T18:04:08.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9279d9b700 1 -- 172.21.15.35:0/3478068842 learned_addr learned my addr 172.21.15.35:0/3478068842 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:08.089 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f927ad9d700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f927c142200 0x7f927c142620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50360/0 (socket says 172.21.15.35:50360) 2022-11-29T18:04:08.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f927a59c700 1 --2- 172.21.15.35:0/3478068842 >> v2:172.21.15.38:3300/0 conn(0x7f927c0aced0 0x7f927c13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9279d9b700 1 -- 172.21.15.35:0/3478068842 >> v2:172.21.15.38:3300/0 conn(0x7f927c0aced0 msgr2=0x7f927c13de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:08.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9279d9b700 1 --2- 172.21.15.35:0/3478068842 >> v2:172.21.15.38:3300/0 conn(0x7f927c0aced0 0x7f927c13de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9279d9b700 1 -- 172.21.15.35:0/3478068842 >> v1:172.21.15.35:6789/0 conn(0x7f927c142200 legacy=0x7f927c142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.090 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9279d9b700 1 -- 172.21.15.35:0/3478068842 --> v2:172.21.15.35:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c069d10 con 0x7f927c099b00 2022-11-29T18:04:08.091 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9279d9b700 1 --2- 172.21.15.35:0/3478068842 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c099b00 0x7f927c13e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f927c09fb50 tx=0x7f9274009840).ready entity=mon.0 client_cookie=225023936d5f27fc server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:08.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f927959a700 1 -- 172.21.15.35:0/3478068842 <== mon.0 v2:172.21.15.35:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927400e070 con 0x7f927c099b00 2022-11-29T18:04:08.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f927959a700 1 -- 172.21.15.35:0/3478068842 <== mon.0 v2:172.21.15.35:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927400ba20 con 0x7f927c099b00 2022-11-29T18:04:08.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f927959a700 1 -- 172.21.15.35:0/3478068842 <== mon.0 v2:172.21.15.35:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927400bda0 con 0x7f927c099b00 2022-11-29T18:04:08.095 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- 172.21.15.35:0/3478068842 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c099b00 msgr2=0x7f927c13e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.096 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 --2- 172.21.15.35:0/3478068842 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c099b00 0x7f927c13e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f927c09fb50 tx=0x7f9274009840).stop 2022-11-29T18:04:08.096 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- 172.21.15.35:0/3478068842 shutdown_connections 2022-11-29T18:04:08.096 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 --2- 172.21.15.35:0/3478068842 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c099b00 0x7f927c13e320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.096 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 --2- 172.21.15.35:0/3478068842 >> v2:172.21.15.38:3300/0 conn(0x7f927c0aced0 0x7f927c13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- 172.21.15.35:0/3478068842 >> 172.21.15.35:0/3478068842 conn(0x7f927c0624c0 msgr2=0x7f927c0ac620 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:08.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- 172.21.15.35:0/3478068842 shutdown_connections 2022-11-29T18:04:08.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.086+0000 7f9280b52700 1 -- 172.21.15.35:0/3478068842 wait complete. 2022-11-29T18:04:08.097 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9280b52700 1 Processor -- start 2022-11-29T18:04:08.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9280b52700 1 -- start start 2022-11-29T18:04:08.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9280b52700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f927c099b00 0x7f927c08ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9280b52700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c0aced0 0x7f927c1413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9280b52700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f927c142200 0x7f927c13e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.098 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9280b52700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c090460 con 0x7f927c0aced0 2022-11-29T18:04:08.099 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9280b52700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c08fb40 con 0x7f927c142200 2022-11-29T18:04:08.099 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9280b52700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0914f0 con 0x7f927c099b00 2022-11-29T18:04:08.099 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f927a59c700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f927c099b00 0x7f927c08ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.099 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f9279d9b700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c0aced0 0x7f927c1413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.100 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f927a59c700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f927c099b00 0x7f927c08ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57862/0 (socket says 172.21.15.35:57862) 2022-11-29T18:04:08.100 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f927a59c700 1 -- 172.21.15.35:0/3049132353 learned_addr learned my addr 172.21.15.35:0/3049132353 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:08.100 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f927ad9d700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f927c142200 0x7f927c13e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.100 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f927a59c700 1 -- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f927c142200 msgr2=0x7f927c13e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.100 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f927a59c700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f927c142200 0x7f927c13e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.101 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.090+0000 7f927a59c700 1 -- 172.21.15.35:0/3049132353 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c0aced0 msgr2=0x7f927c1413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.101 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f927a59c700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c0aced0 0x7f927c1413d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.101 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f927a59c700 1 -- 172.21.15.35:0/3049132353 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009ae0 con 0x7f927c099b00 2022-11-29T18:04:08.102 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f9279d9b700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c0aced0 0x7f927c1413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-29T18:04:08.102 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f927a59c700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f927c099b00 0x7f927c08ff70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f926400e930 tx=0x7f9264009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:08.102 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f92727fc700 1 -- 172.21.15.35:0/3049132353 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264012600 con 0x7f927c099b00 2022-11-29T18:04:08.102 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f92727fc700 1 -- 172.21.15.35:0/3049132353 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264012760 con 0x7f927c099b00 2022-11-29T18:04:08.102 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f92727fc700 1 -- 172.21.15.35:0/3049132353 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264012ae0 con 0x7f927c099b00 2022-11-29T18:04:08.103 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c091810 con 0x7f927c099b00 2022-11-29T18:04:08.103 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c069000 con 0x7f927c099b00 2022-11-29T18:04:08.103 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.094+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927c06b650 con 0x7f927c099b00 2022-11-29T18:04:08.105 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f92727fc700 1 -- 172.21.15.35:0/3049132353 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f9264022070 con 0x7f927c099b00 2022-11-29T18:04:08.105 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f92727fc700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:08.105 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f92727fc700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:08.105 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f92727fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:08.106 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f92727fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:08.106 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f92727fc700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9254084de0 0x7f92540872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.106 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f92727fc700 1 -- 172.21.15.35:0/3049132353 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f92640a8120 con 0x7f927c099b00 2022-11-29T18:04:08.106 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f9279d9b700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9254084de0 0x7f92540872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.107 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.098+0000 7f9279d9b700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9254084de0 0x7f92540872a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f927400c010 tx=0x7f9274009840).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:08.107 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.102+0000 7f92727fc700 1 -- 172.21.15.35:0/3049132353 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f9264050980 con 0x7f927c099b00 2022-11-29T18:04:08.296 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.294+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f927c09e3d0 con 0x7f927c099b00 2022-11-29T18:04:08.364 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.362+0000 7f92727fc700 1 -- 172.21.15.35:0/3049132353 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f926407c170 con 0x7f927c099b00 2022-11-29T18:04:08.369 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.366+0000 7f9280b52700 10 mgrc shutdown 2022-11-29T18:04:08.369 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.366+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9254084de0 msgr2=0x7f92540872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.370 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.366+0000 7f9280b52700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9254084de0 0x7f92540872a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f927400c010 tx=0x7f9274009840).stop 2022-11-29T18:04:08.370 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.366+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f927c099b00 msgr2=0x7f927c08ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.370 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.366+0000 7f9280b52700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f927c099b00 0x7f927c08ff70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f926400e930 tx=0x7f9264009060).stop 2022-11-29T18:04:08.371 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.370+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 shutdown_connections 2022-11-29T18:04:08.371 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.370+0000 7f9280b52700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f9254084de0 0x7f92540872a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.372 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.370+0000 7f9280b52700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f927c142200 0x7f927c13e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.372 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.370+0000 7f9280b52700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f927c0aced0 0x7f927c1413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.372 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.370+0000 7f9280b52700 1 --2- 172.21.15.35:0/3049132353 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f927c099b00 0x7f927c08ff70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.372 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.370+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 >> 172.21.15.35:0/3049132353 conn(0x7f927c0624c0 msgr2=0x7f927c063610 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:08.373 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.370+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 shutdown_connections 2022-11-29T18:04:08.373 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.370+0000 7f9280b52700 1 -- 172.21.15.35:0/3049132353 wait complete. 2022-11-29T18:04:08.385 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2022-11-29T18:04:08.495 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c3fff700 1 Processor -- start 2022-11-29T18:04:08.496 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c3fff700 1 -- start start 2022-11-29T18:04:08.498 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f61c409c460 0x7f61c409bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.498 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c3fff700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40a3250 con 0x7f61c40692c0 2022-11-29T18:04:08.499 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c3fff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40a2f90 con 0x7f61c409a490 2022-11-29T18:04:08.499 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c3fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c40696a0 con 0x7f61c409c460 2022-11-29T18:04:08.499 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c2ffd700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7f61c40692c0 0x7f61c40a2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50368/0 (socket says 172.21.15.35:50368) 2022-11-29T18:04:08.499 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c2ffd700 1 -- 172.21.15.35:0/2968763976 learned_addr learned my addr 172.21.15.35:0/2968763976 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:08.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.494+0000 7f61c37fe700 1 --2- 172.21.15.35:0/2968763976 >> v2:172.21.15.38:3300/0 conn(0x7f61c409c460 0x7f61c409bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c27fc700 1 -- 172.21.15.35:0/2968763976 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515938039 0 0) 0x7f61c40a3250 con 0x7f61c40692c0 2022-11-29T18:04:08.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c27fc700 1 -- 172.21.15.35:0/2968763976 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a40033f0 con 0x7f61c40692c0 2022-11-29T18:04:08.500 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c37fe700 1 -- 172.21.15.35:0/2968763976 >> v1:172.21.15.38:6789/0 conn(0x7f61c409a490 legacy=0x7f61c41398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.501 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c37fe700 1 -- 172.21.15.35:0/2968763976 >> v1:172.21.15.35:6789/0 conn(0x7f61c40692c0 legacy=0x7f61c40a2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.501 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c37fe700 1 -- 172.21.15.35:0/2968763976 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c409ae50 con 0x7f61c409c460 2022-11-29T18:04:08.501 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c37fe700 1 --2- 172.21.15.35:0/2968763976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409c460 0x7f61c409bca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f61bc009e60 tx=0x7f61bc009780).ready entity=mon.1 client_cookie=1b1e07e034ba1d0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:08.501 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c27fc700 1 -- 172.21.15.35:0/2968763976 <== mon.1 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00e070 con 0x7f61c409c460 2022-11-29T18:04:08.502 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c27fc700 1 -- 172.21.15.35:0/2968763976 <== mon.1 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc00bb60 con 0x7f61c409c460 2022-11-29T18:04:08.502 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c27fc700 1 -- 172.21.15.35:0/2968763976 <== mon.1 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc012410 con 0x7f61c409c460 2022-11-29T18:04:08.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c3fff700 1 -- 172.21.15.35:0/2968763976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409c460 msgr2=0x7f61c409bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c3fff700 1 --2- 172.21.15.35:0/2968763976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409c460 0x7f61c409bca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f61bc009e60 tx=0x7f61bc009780).stop 2022-11-29T18:04:08.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c3fff700 1 -- 172.21.15.35:0/2968763976 shutdown_connections 2022-11-29T18:04:08.505 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c3fff700 1 --2- 172.21.15.35:0/2968763976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409c460 0x7f61c409bca0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.506 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c3fff700 1 -- 172.21.15.35:0/2968763976 >> 172.21.15.35:0/2968763976 conn(0x7f61c4061eb0 msgr2=0x7f61c40622c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:08.506 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c3fff700 1 -- 172.21.15.35:0/2968763976 shutdown_connections 2022-11-29T18:04:08.506 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.498+0000 7f61c3fff700 1 -- 172.21.15.35:0/2968763976 wait complete. 2022-11-29T18:04:08.506 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 Processor -- start 2022-11-29T18:04:08.507 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 -- start start 2022-11-29T18:04:08.507 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f61c40692c0 0x7f61c40a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.507 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409a490 0x7f61c4137290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.507 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f61c409c460 0x7f61c414d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.507 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40a49c0 con 0x7f61c409c460 2022-11-29T18:04:08.508 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40a5a20 con 0x7f61c409a490 2022-11-29T18:04:08.508 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4137b30 con 0x7f61c40692c0 2022-11-29T18:04:08.508 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c37fe700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f61c409c460 0x7f61c414d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.508 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f61c40692c0 0x7f61c40a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.509 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f61c40692c0 0x7f61c40a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57872/0 (socket says 172.21.15.35:57872) 2022-11-29T18:04:08.509 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c37fe700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f61c409c460 0x7f61c414d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3300/0 says I am v2:172.21.15.35:43952/0 (socket says 172.21.15.35:43952) 2022-11-29T18:04:08.509 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 -- 172.21.15.35:0/3920626585 learned_addr learned my addr 172.21.15.35:0/3920626585 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:08.509 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61bbfff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409a490 0x7f61c4137290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.509 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 -- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409a490 msgr2=0x7f61c4137290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.510 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409a490 0x7f61c4137290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.510 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 -- 172.21.15.35:0/3920626585 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f61c409c460 msgr2=0x7f61c414d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:08.510 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f61c409c460 0x7f61c414d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:08.510 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 -- 172.21.15.35:0/3920626585 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc009a20 con 0x7f61c40692c0 2022-11-29T18:04:08.511 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c2ffd700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f61c40692c0 0x7f61c40a44d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f61b400c590 tx=0x7f61b4007c80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:08.511 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61baffd700 1 -- 172.21.15.35:0/3920626585 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40072e0 con 0x7f61c40692c0 2022-11-29T18:04:08.511 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61baffd700 1 -- 172.21.15.35:0/3920626585 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4007440 con 0x7f61c40692c0 2022-11-29T18:04:08.511 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c4137df0 con 0x7f61c40692c0 2022-11-29T18:04:08.511 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61baffd700 1 -- 172.21.15.35:0/3920626585 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40057a0 con 0x7f61c40692c0 2022-11-29T18:04:08.512 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.502+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c40a5690 con 0x7f61c40692c0 2022-11-29T18:04:08.513 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c40697e0 con 0x7f61c40692c0 2022-11-29T18:04:08.514 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61baffd700 1 -- 172.21.15.35:0/3920626585 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7f61b401a060 con 0x7f61c40692c0 2022-11-29T18:04:08.514 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61baffd700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:08.514 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61baffd700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:08.514 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61baffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:08.514 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61baffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:08.515 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61baffd700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f61a8084e10 0x7f61a80872d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:08.515 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61baffd700 1 -- 172.21.15.35:0/3920626585 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f61b40a6b20 con 0x7f61c40692c0 2022-11-29T18:04:08.515 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.506+0000 7f61bbfff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f61a8084e10 0x7f61a80872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:08.515 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.510+0000 7f61bbfff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f61a8084e10 0x7f61a80872d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f61c409ef90 tx=0x7f61ac007d40).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:08.516 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.514+0000 7f61baffd700 1 -- 172.21.15.35:0/3920626585 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7f61b407ab70 con 0x7f61c40692c0 2022-11-29T18:04:08.706 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:08.706+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f61c4046830 con 0x7f61c40692c0 2022-11-29T18:04:09.380 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.378+0000 7f61baffd700 1 -- 172.21.15.35:0/3920626585 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f61b4018030 con 0x7f61c40692c0 2022-11-29T18:04:09.384 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 10 mgrc shutdown 2022-11-29T18:04:09.385 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f61a8084e10 msgr2=0x7f61a80872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.385 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f61a8084e10 0x7f61a80872d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f61c409ef90 tx=0x7f61ac007d40).stop 2022-11-29T18:04:09.385 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f61c40692c0 msgr2=0x7f61c40a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.385 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f61c40692c0 0x7f61c40a44d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f61b400c590 tx=0x7f61b4007c80).stop 2022-11-29T18:04:09.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 shutdown_connections 2022-11-29T18:04:09.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7f61a8084e10 0x7f61a80872d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7f61c409c460 0x7f61c414d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61c409a490 0x7f61c4137290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.386 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 --2- 172.21.15.35:0/3920626585 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7f61c40692c0 0x7f61c40a44d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.387 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 >> 172.21.15.35:0/3920626585 conn(0x7f61c4061eb0 msgr2=0x7f61c413b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:09.387 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 shutdown_connections 2022-11-29T18:04:09.388 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.382+0000 7f61c3fff700 1 -- 172.21.15.35:0/3920626585 wait complete. 2022-11-29T18:04:09.402 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-29T18:04:09.512 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb958658700 1 Processor -- start 2022-11-29T18:04:09.512 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb958658700 1 -- start start 2022-11-29T18:04:09.516 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb958658700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb9500632f0 0x7fb95009e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.516 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb958658700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9501309d0 0x7fb950144c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.516 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb958658700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95009ab20 con 0x7fb95009f6b0 2022-11-29T18:04:09.516 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb958658700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95009afb0 con 0x7fb9500632f0 2022-11-29T18:04:09.517 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb958658700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95009ef00 con 0x7fb9501309d0 2022-11-29T18:04:09.517 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb956bf5700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9501309d0 0x7fb950144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:09.517 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.510+0000 7fb956bf5700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9501309d0 0x7fb950144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57888/0 (socket says 172.21.15.35:57888) 2022-11-29T18:04:09.518 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb956bf5700 1 -- 172.21.15.35:0/3660915385 learned_addr learned my addr 172.21.15.35:0/3660915385 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:09.518 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb9563f4700 1 --2- 172.21.15.35:0/3660915385 >> v2:172.21.15.38:3300/0 conn(0x7fb9500632f0 0x7fb95009e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:09.518 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb9553f2700 1 -- 172.21.15.35:0/3660915385 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219478207 0 0) 0x7fb95009ab20 con 0x7fb95009f6b0 2022-11-29T18:04:09.519 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb9553f2700 1 -- 172.21.15.35:0/3660915385 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9380033f0 con 0x7fb95009f6b0 2022-11-29T18:04:09.519 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb956bf5700 1 -- 172.21.15.35:0/3660915385 >> v2:172.21.15.38:3300/0 conn(0x7fb9500632f0 msgr2=0x7fb95009e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:09.519 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb956bf5700 1 --2- 172.21.15.35:0/3660915385 >> v2:172.21.15.38:3300/0 conn(0x7fb9500632f0 0x7fb95009e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.520 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb956bf5700 1 -- 172.21.15.35:0/3660915385 >> v1:172.21.15.38:6789/0 conn(0x7fb95009f6b0 legacy=0x7fb9501302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.520 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb956bf5700 1 -- 172.21.15.35:0/3660915385 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95009f1c0 con 0x7fb9501309d0 2022-11-29T18:04:09.520 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb9563f4700 1 --2- 172.21.15.35:0/3660915385 >> v2:172.21.15.38:3300/0 conn(0x7fb9500632f0 0x7fb95009e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-29T18:04:09.521 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb956bf5700 1 --2- 172.21.15.35:0/3660915385 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9501309d0 0x7fb950144c80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb948009f20 tx=0x7fb948009840).ready entity=mon.2 client_cookie=b560caccf5c44d7f server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:09.522 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb9553f2700 1 -- 172.21.15.35:0/3660915385 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94800e070 con 0x7fb9501309d0 2022-11-29T18:04:09.522 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb9553f2700 1 -- 172.21.15.35:0/3660915385 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94800b970 con 0x7fb9501309d0 2022-11-29T18:04:09.522 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb9553f2700 1 -- 172.21.15.35:0/3660915385 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94800bcf0 con 0x7fb9501309d0 2022-11-29T18:04:09.523 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb958658700 1 -- 172.21.15.35:0/3660915385 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9501309d0 msgr2=0x7fb950144c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.523 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb958658700 1 --2- 172.21.15.35:0/3660915385 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9501309d0 0x7fb950144c80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb948009f20 tx=0x7fb948009840).stop 2022-11-29T18:04:09.523 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb958658700 1 -- 172.21.15.35:0/3660915385 shutdown_connections 2022-11-29T18:04:09.523 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb958658700 1 --2- 172.21.15.35:0/3660915385 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9501309d0 0x7fb950144c80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.524 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb958658700 1 --2- 172.21.15.35:0/3660915385 >> v2:172.21.15.38:3300/0 conn(0x7fb9500632f0 0x7fb95009e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.524 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb958658700 1 -- 172.21.15.35:0/3660915385 >> 172.21.15.35:0/3660915385 conn(0x7fb9500ac620 msgr2=0x7fb9500aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:09.524 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb958658700 1 -- 172.21.15.35:0/3660915385 shutdown_connections 2022-11-29T18:04:09.525 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.514+0000 7fb958658700 1 -- 172.21.15.35:0/3660915385 wait complete. 2022-11-29T18:04:09.530 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 Processor -- start 2022-11-29T18:04:09.530 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 -- start start 2022-11-29T18:04:09.531 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9500632f0 0x7fb95012ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.531 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb95009f6b0 0x7fb950097bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.532 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb9501309d0 0x7fb95012f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.532 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 -- --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500981d0 con 0x7fb9501309d0 2022-11-29T18:04:09.532 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb950098490 con 0x7fb95009f6b0 2022-11-29T18:04:09.533 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 -- --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95012fb60 con 0x7fb9500632f0 2022-11-29T18:04:09.533 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9500632f0 0x7fb95012ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:09.534 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 --2- >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9500632f0 0x7fb95012ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.35:3301/0 says I am v2:172.21.15.35:57890/0 (socket says 172.21.15.35:57890) 2022-11-29T18:04:09.534 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb956bf5700 1 --2- >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb9501309d0 0x7fb95012f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:09.535 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 -- 172.21.15.35:0/1503282723 learned_addr learned my addr 172.21.15.35:0/1503282723 (peer_addr_for_me v2:172.21.15.35:0/0) 2022-11-29T18:04:09.535 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb955bf3700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb95009f6b0 0x7fb950097bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:09.536 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 -- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb95009f6b0 msgr2=0x7fb950097bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.536 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb95009f6b0 0x7fb950097bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.536 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 -- 172.21.15.35:0/1503282723 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb9501309d0 msgr2=0x7fb95012f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.537 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb9501309d0 0x7fb95012f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.537 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 -- 172.21.15.35:0/1503282723 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944009250 con 0x7fb9500632f0 2022-11-29T18:04:09.537 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb955bf3700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb95009f6b0 0x7fb950097bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-29T18:04:09.538 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb956bf5700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb9501309d0 0x7fb95012f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2022-11-29T18:04:09.538 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9563f4700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9500632f0 0x7fb95012ee70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb94400ec80 tx=0x7fb944006f40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:09.538 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9427fc700 1 -- 172.21.15.35:0/1503282723 <== mon.2 v2:172.21.15.35:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944005880 con 0x7fb9500632f0 2022-11-29T18:04:09.538 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9427fc700 1 -- 172.21.15.35:0/1503282723 <== mon.2 v2:172.21.15.35:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9440059e0 con 0x7fb9500632f0 2022-11-29T18:04:09.538 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb948009ae0 con 0x7fb9500632f0 2022-11-29T18:04:09.539 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb9427fc700 1 -- 172.21.15.35:0/1503282723 <== mon.2 v2:172.21.15.35:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944005d60 con 0x7fb9500632f0 2022-11-29T18:04:09.539 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.518+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb950096d70 con 0x7fb9500632f0 2022-11-29T18:04:09.539 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95006b650 con 0x7fb9500632f0 2022-11-29T18:04:09.539 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb9427fc700 1 -- 172.21.15.35:0/1503282723 <== mon.2 v2:172.21.15.35:3301/0 4 ==== mgrmap(e 4) v1 ==== 107408+0+0 (secure 0 0 0) 0x7fb94400ca60 con 0x7fb9500632f0 2022-11-29T18:04:09.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb9427fc700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2022-11-29T18:04:09.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb9427fc700 4 mgrc handle_mgr_map Got map version 4 2022-11-29T18:04:09.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb9427fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:09.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb9427fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] 2022-11-29T18:04:09.540 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb9427fc700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb930084de0 0x7fb9300872a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb9427fc700 1 -- 172.21.15.35:0/1503282723 <== mon.2 v2:172.21.15.35:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb9440a7a80 con 0x7fb9500632f0 2022-11-29T18:04:09.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.522+0000 7fb955bf3700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb930084de0 0x7fb9300872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:09.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.526+0000 7fb955bf3700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb930084de0 0x7fb9300872a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb950068ca0 tx=0x7fb94c007bd0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-29T18:04:09.541 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.530+0000 7fb9427fc700 1 -- 172.21.15.35:0/1503282723 <== mon.2 v2:172.21.15.35:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162146 (secure 0 0 0) 0x7fb94407bb60 con 0x7fb9500632f0 2022-11-29T18:04:09.718 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.718+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9500944a0 con 0x7fb9500632f0 2022-11-29T18:04:09.719 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.718+0000 7fb9427fc700 1 -- 172.21.15.35:0/1503282723 <== mon.2 v2:172.21.15.35:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3724 (secure 0 0 0) 0x7fb944016070 con 0x7fb9500632f0 2022-11-29T18:04:09.720 INFO:teuthology.orchestra.run.smithi035.stdout: 2022-11-29T18:04:09.720 INFO:teuthology.orchestra.run.smithi035.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4196,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.35:6811/336810684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6810","nonce":336810684},{"type":"v1","addr":"172.21.15.35:6811","nonce":336810684}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4202,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6827/2614917468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2614917468},{"type":"v1","addr":"172.21.15.38:6827","nonce":2614917468}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2022-11-29T18:04:07.355264+0000","modified":"2022-11-29T18:04:09.368948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4215":{"gid":4215,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.35:6809/2456090531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.35:6808","nonce":2456090531},{"type":"v1","addr":"172.21.15.35:6809","nonce":2456090531}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2022-11-29T18:04:09.723 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 10 mgrc shutdown 2022-11-29T18:04:09.723 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb930084de0 msgr2=0x7fb9300872a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.724 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb930084de0 0x7fb9300872a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fb950068ca0 tx=0x7fb94c007bd0).stop 2022-11-29T18:04:09.724 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9500632f0 msgr2=0x7fb95012ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.724 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9500632f0 0x7fb95012ee70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb94400ec80 tx=0x7fb944006f40).stop 2022-11-29T18:04:09.725 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 shutdown_connections 2022-11-29T18:04:09.725 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:6824/24307,v1:172.21.15.38:6825/24307] conn(0x7fb930084de0 0x7fb9300872a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.726 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fb9501309d0 0x7fb95012f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.726 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb95009f6b0 0x7fb950097bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.727 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 --2- 172.21.15.35:0/1503282723 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fb9500632f0 0x7fb95012ee70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.727 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 >> 172.21.15.35:0/1503282723 conn(0x7fb9500ac620 msgr2=0x7fb95009d300 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-29T18:04:09.728 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 shutdown_connections 2022-11-29T18:04:09.728 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.722+0000 7fb958658700 1 -- 172.21.15.35:0/1503282723 wait complete. 2022-11-29T18:04:09.729 INFO:teuthology.orchestra.run.smithi035.stderr:dumped fsmap epoch 6 2022-11-29T18:04:09.733 DEBUG:teuthology.orchestra.run.smithi035:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-29T18:04:09.838 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c3fff700 1 Processor -- start 2022-11-29T18:04:09.838 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c3fff700 1 -- start start 2022-11-29T18:04:09.841 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c3fff700 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa9c4067320 0x7fa9c4136ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.842 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c3fff700 1 -- --> v1:172.21.15.35:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c41378b0 con 0x7fa9c41373c0 2022-11-29T18:04:09.842 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c3fff700 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c4137b50 con 0x7fa9c409f130 2022-11-29T18:04:09.842 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c3fff700 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c40693c0 con 0x7fa9c4067320 2022-11-29T18:04:09.842 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c37fe700 1 --1- >> v1:172.21.15.35:6789/0 conn(0x7fa9c41373c0 0x7fa9c41418f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.35:6789/0 says I am v1:172.21.15.35:50378/0 (socket says 172.21.15.35:50378) 2022-11-29T18:04:09.843 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c37fe700 1 -- 172.21.15.35:0/3833829803 learned_addr learned my addr 172.21.15.35:0/3833829803 (peer_addr_for_me v1:172.21.15.35:0/0) 2022-11-29T18:04:09.843 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c27fc700 1 --2- 172.21.15.35:0/3833829803 >> v2:172.21.15.38:3300/0 conn(0x7fa9c4067320 0x7fa9c4136ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2022-11-29T18:04:09.843 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 <== mon.0 v1:172.21.15.35:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164144874 0 0) 0x7fa9c41378b0 con 0x7fa9c41373c0 2022-11-29T18:04:09.843 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 --> v1:172.21.15.35:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a40033f0 con 0x7fa9c41373c0 2022-11-29T18:04:09.844 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 <== mon.1 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60117961 0 0) 0x7fa9c4137b50 con 0x7fa9c409f130 2022-11-29T18:04:09.844 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c41378b0 con 0x7fa9c409f130 2022-11-29T18:04:09.844 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 <== mon.0 v1:172.21.15.35:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1430160832 0 0) 0x7fa9a40033f0 con 0x7fa9c41373c0 2022-11-29T18:04:09.844 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 >> v2:172.21.15.38:3300/0 conn(0x7fa9c4067320 msgr2=0x7fa9c4136ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-29T18:04:09.845 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 --2- 172.21.15.35:0/3833829803 >> v2:172.21.15.38:3300/0 conn(0x7fa9c4067320 0x7fa9c4136ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2022-11-29T18:04:09.845 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 >> v1:172.21.15.38:6789/0 conn(0x7fa9c409f130 legacy=0x7fa9c4068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.845 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 --> v1:172.21.15.35:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c4069500 con 0x7fa9c41373c0 2022-11-29T18:04:09.845 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 <== mon.0 v1:172.21.15.35:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2443281136 0 0) 0x7fa9b4001470 con 0x7fa9c41373c0 2022-11-29T18:04:09.845 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 >> v1:172.21.15.35:6789/0 conn(0x7fa9c41373c0 legacy=0x7fa9c41418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-29T18:04:09.846 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 --2- 172.21.15.35:0/3833829803 >> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] conn(0x7fa9a4003a70 0x7fa9a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.846 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 --2- 172.21.15.35:0/3833829803 >> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] conn(0x7fa9a4004690 0x7fa9a4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.846 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 --2- 172.21.15.35:0/3833829803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9a4005000 0x7fa9a400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2022-11-29T18:04:09.846 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 --> [v2:172.21.15.35:3300/0,v1:172.21.15.35:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40693c0 con 0x7fa9a4003a70 2022-11-29T18:04:09.847 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a400be80 con 0x7fa9a4005000 2022-11-29T18:04:09.847 INFO:teuthology.orchestra.run.smithi035.stderr:2022-11-29T18:04:09.838+0000 7fa9c1ffb700 1 -- 172.21.15.35:0/3833829803 --> [v2:172.21.15.35:3301/0,v1:172.21.15.35:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a400c040 con 0x7fa9a4004690 2022-11-29T18:04:09.847 INFO:teuth